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REAL PARTY IN INTEREST 

The present application has been assigned to Parus Holdings, Inc., a corporation 

organized under the laws of the State of Delaware, having a principal place of business in 
Bannockburn, Illinois. 

RELATED APPEALS AND INTERFERENCES 

No appeals or interferences are related to the present application. 

STATUS OF CLAIMS 

Claims 32, 34 and 50-68 are pending in the present application. Claims 32-34 and 50-68 

have been rejected under 35 U.S.C. § 102(e) as being anticipated by U.S. Patent Application 
Publication No. 2001/0032234 to Summers et al. 

Claims 1-31, 33 and 35-49 previously were canceled. No claims have been withdrawn, 
objected to, confirmed or allowed. 

Claims 32, 34 and 50-68 are on appeal. 

STATUS OF AMENDMENTS 

No amendments are pending with respect to the present application. 
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SUMMARY OF CLAIMED SUBJECT MATTER 

Following is a summary of the claimed subject matter, as set forth in independent claims 32, 

53, and 63, with reference to specific examples disclosed in the specification and shown in the 
figures. 

I. CLAIM 32 

Claim 32 claims a method for allowing users to use speech commands to obtain 
information from a pre-defined portion of a pre-selected web site in audio format (Specification, 
U 18). Claim 32 calls for providing a computer which has a speech processor and which is 
operatively connected to the internet and to at least one phone (Specification, ][ 19; Fig. 3). Next, 
a URL, which indicates a pre-selected web site from which the information is to be retrieved, is 
provided to the computer (Specification, ^ 21; Fig. 2). Then, using the computer, a pre-defined 
portion of the pre-selected web site which contains the information to be retrieved is designated 
(Specification, ^21; Fig. 2). Next, using the computer, a named object associated with the 
content of the information to be retrieved is identified (Specification, ^21; Fig. 2). Then, the 
computer creates a descriptor containing instructions which identify the web site URL, the 
location of the pre-defined portion of said pre-selected web site which contains the information 
to be retrieved, and the named object (Specification, ][21; Fig. 4). A user provides a speech 
command corresponding to the descriptor to the speech processor (Specification, 23; Fig. 4). 
Then, the speech processor converts the speech command to a digital-form command 
(Specification, ^27). The computer receives the digital-form command from the speech 
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processor, and the computer assigns the descriptor to the digital-form command (Specification, 
1127). 

Once the system has been set up, as described above, the user transmits an audio speech 
command corresponding to the descriptor to the speech processor (Specification, ]f 36). The 
speech processor converts the audio speech command into the digital-form command 
(Specification, ]f 37). The computer then receives the digital-form command from the speech 
processor (Specification, ]f 38). Next, the computer retrieves the descriptor corresponding to the 
digital-form command (Specification, ]f 38). Then, the computer retrieves the information from 
the pre-defined portion of the pre-selected web site corresponding to the descriptor when the 
requested information is found in the pre-defined portion of the pre-selected web site 
(Specification, ]f 39). When the requested information is not found in the pre-defined portion of 
the pre-selected web site, the computer searches the pre-selected web site for the named object 
(Specification, fflf 32, 39). Then, the computer provides the retrieved information to the speech 
processor (Specification, ]f 40). The speech processor then converts the retrieved information 
into an audio message (Specification, ]f 40). Finally, the speech processor forwards the audio 
message to the user (Specification, flf 35, 40). 

II. CLAIM 53 

Claim 53 claims a system for retrieving information from a pre-defined portion of a pre- 
selected web site by uttering speech commands into a phone and for providing to a user retrieved 
information in an audio form (Specification, 18, 24). Claim 53 calls for a server being 
operatively connected to the internet and to at least one phone (Specification, fflf 19, 26; Figs. 3, 
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5). The server includes telephony hardware being operatively connected to the phone and to the 
server (Specification, 1 26; Figs. 3, 5). The server also includes at least one speech recognition 
engine being operatively connected to the server and to the telephony hardware (Specification, 
| 27; Figs. 3, 5). The server further includes a speech synthesis engine being operatively 
connected to the server and to the telephony hardware (Specification, 1 29; Figs. 3, 5). The 
server still further includes a call processing system, which is configured to receive speech 
commands through the telephony hardware and to forward the speech commands to the speech 
recognition engine and is further configured to receive an audio message from the speech 
synthesis engine and to forward the audio message through the telephony hardware 
(Specification, 1 27; Figs. 3, 5). The system further includes at least one instruction set stored on 
the server for identifying the pre-defined portion of the pre-selected web site, which contains the 
information to be retrieved, and for identifying a named object associated with the content of the 
information to be retrieved (Specification, 1 20; Fig. 4). Each instruction set includes a uniform 
resource locator address for the web site (Specification, 1 21 ; Fig. 4). Each instruction set also 
includes a content descriptor of the web site, which pre-defines the portion of the web site from 
where the information is to be retrieved (Specification, || 21, 31; Table 1; Figs. 2, 4). Each 
instruction set further includes the named object (Specification, 1 32). The system further 
includes a recognition grammar corresponding to each instruction set and corresponding to a 
speech command (Specification, 1 23). The speech recognition engine is configured to receive 
the speech command, to select the corresponding recognition grammar, and to retrieve each 
instruction set corresponding to the recognition grammar upon receiving the speech command 
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(Specification, ]f 23). The system further includes a web browser, which includes at least a 
content extraction agent, a content fetcher, and a content descriptor file, being operatively 
connected to the server and being configured to access the pre-defined portion of the web site 
defined by the instruction set and to retrieve the information defined by the instruction set 
(Specification, ]f 30). The system further includes a said speech synthesis engine which is 
configured to convert the retrieved information from the pre-defined portion of the pre-selected 
web site into an audio message and to transmit the audio message to the user (Specification, 
135). 

III. CLAIM 63 

Claim 63 claims a method for allowing a phone user to set up and subsequently retrieve 
information in an audio format from a pre-defined portion of a pre-selected web site 
(Specification, ]f 1 8). Claim 63 calls for providing a server being operatively connected to the 
internet and to at least one phone and being further operatively connected to a speech recognition 
engine and to a speech synthesis engine (Specification, Tffl 19, 26; Fig. 5). Next, a user provides 
at least one instruction set stored on the server for identifying the pre-defined portion of a pre- 
selected web site containing the information to be retrieved from the web site (Specification, 
Tit 2 1 , 3 1 ; Table 1 ; Figs. 2, 4). Each instruction set includes a uniform resource locator address 
for the web site (Specification, Tf 21). Each instruction set further includes a content descriptor of 
the web site, which pre-defines the portion of the web site from which the information is to be 
retrieved (Specification, Tflf 21, 31; Table 1; Figs. 2, 4). Each instruction set still further includes 
the named object (Specification, ]f 32). Then, the user provides a speech command to the speech 
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recognition engine which corresponds to the instruction set (Specification, ]j 23). The speech 
recognition engine then assigns the speech command to a recognition grammar, both of which 
correspond to the instruction set (Specification, ]f 24). 

Then, the user transmits the speech command to the speech recognition engine 
(Specification, ]j 36). The speech recognition engine receives the speech command and selects 
the corresponding recognition grammar (Specification, ^ 37). Then, the server retrieves each 
instruction set corresponding to the recognition grammar (Specification, ]f 37). The server then 
accesses the pre-defined portion of the pre-selected web site defined by the instruction set and 
retrieves the information defined by the instruction set when the requested information is found 
in the pre-defined portion of the pre-selected web site (Specification, | 38). When the requested 
information is not found in the pre-defined portion of the pre-selected web site, the server 
searches the pre-selected website (Specification, Iflj 32, 38). Then, the speech synthesis engine 
converts the retrieved information from the pre-selected web site into an audio message 
(Specification, Tf 39. Finally, the speech synthesis engine transmits the audio message to the user 
(Specification, 35, 39). 

GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 

1. Whether Claims 32-34 and 50-68 are unpatentable under 35 U.S.C. § 102(e) 

("Section 102(e)") as being anticipated by U.S. Patent Application Publication No. 2001/0032234 
to Summers et al. ("the Summers Application"). 
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ARGUMENT 

I. SUMMARY 

Applicant traverses the final rejection of claims 32-34 and 50-67 under Section 102(e) as 
being anticipated by the Summers Application. The Summers Application is not prior art with 
respect to the rejected claims of the present application. The Summers Application was filed 
after the present application. The parent applications to which the Summers Application claims 
priority do not disclose all of the elements of Applicant's claimed invention and, therefore, 
cannot form the basis for a Section 102(e) rejection. Accordingly, Applicant requests that the 
Board reverse the final rejection based on the Summers Application. 

The relationship of these applications is shown in Figure 1 below: 
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FIGURE 1 



Specifically, Applicant filed the present application on February 6, 2001 . More than three 
months later, on May 1 1, 2001, the Summers Application was filed. The Summers Application 
claims priority to U.S. Provisional Patent Application No. 60/263,003 ("the '003 Provisional 



CHICJ626626.1 



Atty. Dkt. No. 078616-0128 

Application") filed January 19, 2001 and U.S. Patent Application No. 09/464,989 to Wesemann 
et al. ("the Wesemann Application") filed December 16, 1999. However, neither the '003 
Provisional Application or the Wesemann Application contain the pertinent disclosure relied on 
by the Examiner for the Section 102(e) rejection. Therefore, the effective filing date for the later- 
added disclosure is the May 11, 2001 filing date of the Summers Application. In addition, the 
later-added disclosure relied on by the Examiner does not disclose all limitations of the rejected 
claims, as required for a Section 102 rejection. 

Thus, the final rejection of the present application should be reversed for two reasons. 
First, the Summers Application was filed more than three months after the present application 
and is not prior art with respect to the claimed invention. 

Second, as a matter of law the pertinent disclosure first added in the Summers Application 
is not entitled to an effective filing date of either the '003 Provisional Application or the 
Wesemann Application, because these parent applications do not include the pertinent disclosure 
and, therefore, cannot be used to form the basis for a Section 102(e) rejection. 

Accordingly, for the reasons stated herein, Applicant respectfully requests that the Board 
reverse the final rejection of the present application. 

II. THE SUMMERS APPLICATION IS NOT PRIOR ART 

With respect to subject matter disclosed for the first time in the Summers Application, the 
effective filing date is the same as the filing date of the Summers Application, May 11, 2001. As 
the Summers Application was filed more than three months after Applicant's February 6, 2001 
filing date, the Summers Application is not prior art with respect to the claimed invention. 

-9- 
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A. With respect to subject matter disclosed for the first time in the Summers 
Application, the effective filing date is the same as the filing date of the 
Summers Application. 

As a matter of law, any subject matter disclosed in the Summers Application that is not 
supported by either the Wesemann Application or the '003 Provisional Application is not entitled 
to an effective filing date prior to the May 1 1, 2001 filing date of the Summers Application. 
Because the Summers Application cannot pre-date Applicant, the Summers Application cannot be 
considered Section 102(e) prior art with respect to the present application. See MPEP § 201.1 1 
and cases cited in Section III(B) infra. Accordingly, with respect to the present invention, the 
effective filing date of the pertinent disclosure first added in the Summers Application is the 
same as the Summer Application's filing date — May 1 1 , 2001 . 

In addition, the later-added pertinent disclosure relied on by the Examiner does not 
disclose all limitations of the rejected claims, as required for a Section 102 rejection. Applicant's 
claimed invention is different because it locates information contained on a webpage based on a 
comprehensive description of the requested information, which is contained in a content 
descriptor file. In the present application, the novel content descriptor file is claimed in 
independent claims 32, 53, and 63. Specifically, independent claims 32, 53, and 63 claim an 
instruction set which includes the content descriptor file: 

at least one instruction set stored on said server for identifying the 
pre-defined portion of the pre-selected web site and for identifying 
a named object associated with the content of the information to be 
retrieved, said pre-defined portion containing the information to be 
retrieved from the web site, each said instruction set comprising: a 
uniform resource locator address for said web site; a content 
descriptor of said web site, said content descriptor pre-defining 
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the portion of said web site from which said information is to be 
retrieved, and the named object. 

(emphasis added). 

This claimed feature is not disclosed in the Summers Application. Specifically, the 
Summers Application does not disclose "a content descriptor of said web site, said content 
descriptor pre-defining the portion of said web site from which said information is to be 
retrieved, and the named object." Rather, the systems of the Summers Application only locates 
information embedded within a specific region mapped by the user and does not search for 
requested information based on a content descriptor file, or a comprehensive description of the 
requested information. Summers Application, fflf 79-80. 

Accordingly, the pertinent disclosure relied on by the Examiner, which was first added to 
the Summers Application, does not disclose every element of the claimed invention and therefore 
cannot support a rejection under Section 102. 

B. The filing date of the Summers Application is more than three months after 
Applicant's filing date, so the Summers Application cannot be prior art with 
respect to Applicant. 

Even if the Summers Application discloses every element of the claimed invention, any 

such disclosure cannot claim an effective filing date prior to the May 11, 2001 filing date of the 

Summers Application, because neither the '003 Provisional Application or the Wesemann 

Application include the pertinent disclosure, as described in greater detail in Section III(A) infra. 

As the May 11, 2001 filing date of the Summers Application is more than three months after 

Applicant's February 6, 2001 filing date, any disclosure of this claimed feature by the Summers 

Application is not prior art with respect to the present application. 

-11- 

CHICJ626626.1 



Atty. Dkt. No. 078616-0128 

Accordingly, the Summers Application is not prior art with respect to the present 
application. 

III. BECAUSE THE PERTINENT DISCLOSURE IS NOT DISCLOSED IN THE '003 
PROVISIONAL APPLICATION OR THE WESEMANN APPLICATION BUT 
RATHER WAS FIRST ADDED IN THE SUMMERS APPLICATION, IT IS NOT 
ENTITLED TO AN EARLIER EFFECTIVE FILING DATE 

The '003 Provisional Application and the Wesemann Application do not include the 

pertinent disclosure of the Summers Application but rather disclose a different method of 

locating information. The pertinent disclosure included in Summers Application, which was 

relied on by the Examiner for the Section 102(e) rejection, was first added in the Summers 

Application. 

Therefore, as a matter of law the pertinent disclosure first added in the Summers 
Application is not entitled to an effective filing date earlier than the filing date of the Summers 
Application. 

A. The '003 Provisional Application and the Wesemann Application disclose a 
different method of locating information and therefore they do not disclose 
the claimed invention. 

Although the '003 Provisional Application and the Wesemann Application disclose 

methods of locating information in documents, they do not disclose the same method as 

described and claimed by Applicant. Rather, the '003 Provisional Application and the 

Wesemann Application disclose methods that only search certain specific portions of documents 

and therefore only can locate information contained within those specific portions of the 

documents. By contrast, Applicant discloses and claims a system and method for locating and 

retrieving information contained anywhere on a webpage based on comprehensive instructions 
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describing the requested information such that the information can be located regardless of its 
location on the webpage. 

The system of the '003 Provisional Application only locates information embedded 
within pointers contained in a document. With the system of the '003 Provisional Application, a 
user creates start and end pointers. '003 Provisional Application, p. 6, lines 10-11. The system 
of the '003 Provisional searches a document by locating the pointers and searching the 
information located within a certain set of pointers. '003 Provisional Application, p. 6, lines 17- 
18. As a result, only information embedded within the pointers is located when the system of the 
'003 Provisional Application searches a document. 

Similarly, the system of the Wesemann Application only locates information embedded 
within HTML tags contained in a document. As clearly shown in Fig. 3, the system of the 
Wesemann Application only accepts and processes HTML information. See Fig. 3 (showing 
HTML Source 310 as the only input); see also Wesemann Application, p. 13, lines 14-17 ("Voice 
Browser 340 provides an audio interface to HTML Source 310 that is suitable for use by 
Telephone 350."). The system of the Wesemann Application searches the HTML information by 
reading the HTML tags. Wesemann Application, p. 17, lines 11-17 ("The enhanced mapping 
feature of the present invention for My Yahoo! Pages looks for a tag with a particular 
background color attribute. Other mappings may use other HTML tags and/or tag attributes to 
identify categories."). As a result, only information embedded within the HTML tags is located 
when the system of the Wesemann Application searches a document. 
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In contrast, Applicant's claimed invention is different because it locates information 
contained on a webpage based on a comprehensive description of the requested information, 
which is contained in a content descriptor file. Applicant's content descriptor file includes other 
aspects of a webpage beyond pointers such as HTML tags. 

For instance, Table 1 of the Specification shows an example of a comprehensive content 
descriptor file used by Applicant's system to locate requested information from a webpage. As 
shown in Table 1 , the content descriptor file includes numerous information, including the web 
address, the pre-defined portion of the webpage where the information is likely to be found, and a 
comprehensive list of named objects, or specific identifiers describing the information to be 
retrieved. As the example in Table 1 relates to retrieving current weather conditions, the content 
descriptor file includes several named objects relating to weather conditions, including the 
forecast, current temperature, temperature highs and lows, wind speed, barometric pressure, and 
the time of sunrise and sunset. Based on this content descriptor file, Applicant's invention can 
retrieve various information relating to the current weather conditions from a webpage. 
Application, ^ 3 1 . To the extent that the requested information is not located on the webpage at 
the location specified by the content descriptor file, Applicant's invention searches the webpage 
to locate information relating to the requested information based on the named objects, or the 
description of the requested information set forth in the content descriptor file. Application, 
If 32. 
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In the present application, the novel content descriptor file is claimed in independent 
claims 32, 53, and 63. Specifically, independent claims 32, 53, and 63 claim an instruction set 
which includes the content descriptor file: 

at least one instruction set stored on said server for identifying the 
pre-defined portion of the pre-selected web site and for identifying 
a named object associated with the content of the information to be 
retrieved, said pre-defined portion containing the information to be 
retrieved from the web site, each said instruction set comprising: a 
uniform resource locator address for said web site; a content 
descriptor of said web site, said content descriptor pre-defining 
the portion of said web site from which said information is to be 
retrieved, and the named object. 

(emphasis added). 

This claimed feature is not disclosed in the '003 Provisional Application and the 
Wesemann Application. Specifically, these applications do not disclose "a content descriptor of 
said web site, said content descriptor pre-defining the portion of said web site from which said 
information is to be retrieved, and the named object." Rather, as set forth above, the systems of 
the '003 Provisional Application and the Wesemann Application only locate information 
embedded within pointers, such as HTML tags, and do not search for requested information 
based on a content descriptor file, or a comprehensive description of the requested information. 

Accordingly, the '003 Provisional Application and the Wesemann Application do not 
disclose the claimed invention but rather disclose a different method of locating information in a 
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B. The pertinent disclosure was first disclosed in the Summers Application and 
under well-established law is not entitled to an effective filing date prior to 
the filing date of the Summers Application. 

Because the '003 Provisional Application and the Wesemann Application do not include 
the pertinent disclosure but rather disclose a different method of locating information in a 
document, as a matter of law the pertinent disclosure first added in the Summers Application is 
not entitled to an earlier effective filing date than the filing date of the Summers Application. 

The Summers Application is a continuation-in-part of the Wesemann Application, which 
means that it discloses additional information than that disclosed by the Wesemann Applicaiton. 
Subject matter disclosed for the first time in a continuation-in-part application such as the 
Summers Application that is not supported by the parent application {i.e., Wesemann 
Application) is only entitled to the filing date of the continuation-in-part application and is not 
entitled to the filing date of the earlier-filed parent application. Transco Prods., Inc. v. 
Performance Contracting Inc., 38 F.3d 551, 557, 32 U.S.P.Q.2d 1077 (Fed. Cir. 1994) ("an 
application is entitled to the benefit of the filing date of an earlier application as to common 
subject matter"); In re Van Lagenhoven, 458 F.2d 132, 136, 173 U.S.P.Q. 426 (C.C.P.A. 1972) 
("subject matter which is first disclosed in a continuation-in-part application is not entitled to the 
filing date of the parent application"); MPEP § 201.1 1(I)(B) ("[a claim] which was first 
introduced or adequately supported in the continuation-in-part application ... is entitled only to 
the filing date of the continuation-in-part application"); MPEP § 2133.01 ("When applicant files 
a continuation-in-part whose claims are not supported by the parent application, the effective 
filing date is the filing date of the child CIP."). In other words, with respect to the pertinent 
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disclosure first added in the Summers Application, the effective filing date is the same as the 
filing date of the Summers Application and is not entitled to the filing date of the Wesemann 
Application. 

The Summers Application also is a non-provisional application of the '003 Provisional 
Application. Similar to continuation-in-part applications, subject matter disclosed for the first 
time in a non-provisional application such as the Summers Application that is not supported by 
an earlier-filed provisional application (i.e., '003 Provisional Application) is entitled only to the 
filing date of the non-provisional application. New Railhead Mfg., L.L.C. v. Vermeer Mfg. Co., 
298 F.3d 1290, 1294, 63 U.S.P.Q.2d 1843 (Fed. Cir. 2002); MPEP § 201.1 1(I)(A) ("If 
[disclosure] in the nonprovisional application is not adequately supported by the written 
description and drawing(s) (if any) of the provisional application . . . [the disclosure] is not 
entitled to the benefit of the filing date of the provisional application.") (emphasis in original). In 
other words, with respect to the pertinent disclosure first disclosed in the Summers Application, 
the effective filing date is the same as the filing date of the Summers Application and is not 
entitled to the filing date of the '003 Provisional Application. 

Based on this well-established law, the pertinent disclosure first added in the Summers 
Application is not entitled to the filing date of the '003 Provisional Application or the Wesemann 
Application but rather has an effective filing date the same as the filing date of the Summers 
Application. As reflected in Figure 1 above, the Summers Application includes additional 
pertinent disclosure that is not included in either the '003 Provisional Application or the 
Wesemann Application. For example, the Summers Application includes additional paragraphs, 
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including at least ^ 78-81, which are not present in either the '003 Provisional Application or 
the Wesemann Application. This additional pertinent disclosure, including^ 78-81, is 
important, because it serves as part of the basis of the Examiner's final rejection. See Final 
Office Action dated March 19, 2007, at pp. 3-5 (showing numerous references to some of these 
paragraphs in the Examiner's rejection of the independent claims). 

Because the Summers Application includes additional pertinent disclosure that is not 
included in either the '003 Provisional Application or the Wesemann Application, the effective 
filing date of the additional pertinent disclosure is the same as the filing date of the Summers 
Application. Accordingly, the later-filed pertinent disclosure first added in the Summers 
Application is not entitled to the filing date of the '003 Provisional Application or the Wesemann 
Application. 

IV. CONCLUSION 

In view of the foregoing, Applicant respectfully requests that the Board reverse the final 
rejection of claims 32-34 and 50-67 and allow the application to issue, because the Summers 
Application, which was filed after the present application, is not prior art and the parent 
applications to which the Summers Application claims priority — as necessary to support a 
Section 102(e) rejection — do not disclose all of the elements of Applicant's claimed invention. 
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CLAIMS APPENDIX 

Claims 1-31 (Canceled). 

Claim 32 (Previously Presented): A method for allowing users to use speech commands 
to obtain information from a pre-defined portion of a pre-selected web site in audio format, said 
method comprising the steps of: 

(a) providing a computer having a speech processor, said computer being operatively 
connected to the internet and to at least one phone; 

(b) providing a URL to said computer, said URL indicating a pre-selected web site 
from which the information is to be retrieved; 

(c) using said computer to designate a pre-defined portion of the pre-selected web site 
which contains the information to be retrieved; 

(d) using said computer to identify a named object associated with the content of the 
information to be retrieved; 

(e) allowing said computer to create a descriptor containing instructions which 
identify the web site URL, the location of the pre-defined portion of said pre-selected web site 
which contains said information to be retrieved, and said named object; 

(f) providing a speech command to said speech processor, said speech command 
corresponding to said descriptor; 

(g) said speech processor converting said speech command to a digital-form 
command; 

(h) said computer receiving said digital-form command from said speech processor, 
said computer assigning said descriptor to said digital-form command; 
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(i) after steps (a) through (h) are completed, transmitting an audio speech command 
to said speech processor, said speech command corresponding to said descriptor; 

G) said speech processor converting said audio speech command to said digital-form 
command; 

(k) said computer receiving said digital-form command from said speech processor; 

(1) said computer retrieving said descriptor corresponding to said digital-form 
command; 

(m) said computer retrieving the information from the pre-defined portion of the pre- 
selected web site corresponding to said descriptor when the requested information is found in the 
pre-defined portion of the pre-selected web site; 

(n) said computer searching said pre-selected web site for said named object when the 
requested information is not found in the pre-defined portion of the pre-selected web site; 

(o) said computer providing said retrieved information to said speech processor; 

(p) said speech processor converting said retrieved information into an audio 
message; and 

(q) said speech processor forwarding said audio message to a user. 
Claim 33 (Canceled). 

Claim 34 (Previously Presented): The method of claim 32 wherein the pre-defined 
portion of the pre-selected web site being retrieved is periodically updated. 

Claims 35-49 (Canceled). 

Claim 50 (Previously Presented): The method of claim 32 wherein the step of providing 
a URL to a computer is performed by a user. 
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Claim 51 (Previously Presented): The method of claim 32 wherein the step of using said 
computer to designate a pre-defined portion of the web site which contains the information to be 
retrieved comprises the steps of: 

displaying the web site on a graphical display operatively connected to the computer; and 

using computer software to select the pre-defined portion of the pre-selected web site 
which contains the information to be retrieved. 

Claim 52 (Previously Presented): The method of claim 51 wherein the step of using said 
computer to designate a pre-defined portion of the web site which contains the information to be 
retrieved is performed by a user. 

Claim 53 (Previously Presented): A system for retrieving information from a pre-defined 
portion of a pre-selected web site by uttering speech commands into a phone and for providing to 
a user retrieved information in an audio form, said system comprising: 

a server, said server operatively connected to the internet and to at least one phone, said 
server comprising: 

telephony hardware, said telephony hardware operatively connected to said phone 
and to said server; 

at least one speech recognition engine, said speech recognition engine operatively 
connected to said server and to said telephony hardware; 

a speech synthesis engine, said speech synthesis engine operatively connected to 
said server and to said telephony hardware; and 

a call processing system, said call processing system configured to receive speech 
commands through said telephony hardware and forward said speech commands to said 
speech recognition engine and said call processing system further configured to receive 
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an audio message from said speech synthesis engine and forward said audio message 
through said telephony hardware; 

at least one instruction set stored on said server for identifying the pre-defined portion of 
the pre-selected web site and for identifying a named object associated with the content of the 
information to be retrieved, said pre-defined portion containing the information to be retrieved 
from the web site, each said instruction set comprising: 

a uniform resource locator address for said web site; 

a content descriptor of said web site, said content descriptor pre-defining the 
portion of said web site from which said information is to be retrieved, and 

the named object; 

a recognition grammar corresponding to each said instruction set and corresponding to a 
speech command; 

said speech recognition engine configured to receive said speech command and to select 
the corresponding recognition grammar, said speech recognition engine further configured to 
retrieve each said instruction set corresponding to said recognition grammar upon receiving said 
speech command; 

a web browser operatively connected to said server, said web browser including at least a 
content extraction agent, a content fetcher, and a content descriptor file, said web browser 
configured to access said pre-defined portion of said web site defined by said instruction set and 
to retrieve said information defined by said instruction set; 

said speech synthesis engine configured to convert the retrieved information from said 
pre-defined portion of said pre-selected web site into an audio message, and said speech 
synthesis engine further configured to transmit said audio message to said user. 
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Claim 54 (Previously Presented): The system of claim 53 wherein the phone is a landline 
telephone. 

Claim 55 (Previously Presented): The system of claim 53 wherein the phone is a wireless 
telephone. 

Claim 56 (Previously Presented): The system of claim 53 wherein the phone is an 
internet protocol telephone. 

Claim 57 (Previously Presented): The system of claim 53 wherein the server is 
operatively connected to a local area network. 

Claim 58 (Previously Presented): The system of claim 53 wherein the server is 
operatively connected to a wide area network. 

Claim 59 (Previously Presented): The system of claim 53 wherein the server is 
operatively connected to the Internet. 

Claim 60 (Previously Presented): The system of claim 53 further comprising a database 
operatively connected to the server, the database configured to store said instruction set and said 
recognition grammars. 

Claim 61 (Previously Presented): The system of claim 53 further comprising computer 
software stored on the server, said computer software configured to create said instruction set 
based on user-defined information. 

Claim 62 (Previously Presented): The system of claim 53 further comprising: 

a graphical display operatively connected to the server, said graphical display configured 
to display the pre-selected web site; and 

computer software stored on the server, said computer software configured to select the 
pre-defined portion of the pre-selected web site which contains the information to be retrieved. 
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Claim 63 (Previously Presented): A method for allowing a phone user to set up and 
subsequently retrieve information in an audio format from a pre-defined portion of a pre-selected 
web site, said method comprising the steps of: 

providing a server operatively connected to the internet and to at least one phone, said 
server being operatively connected to a speech recognition engine and to a speech synthesis 
engine; 

providing at least one instruction set stored on said server for identifying the pre-defined 
portion of a pre-selected web site containing the information to be retrieved from the web site, 
each said instruction set comprising: 

a uniform resource locator address for said web site; and 

a content descriptor of said web site, said content descriptor defining the portion 
of said web site from which said information is to be retrieved; and 

a named object associated with the content of the information to be retrieved; 

providing a speech command to said speech recognition engine, said speech command 
corresponding to said instruction set; 

said speech recognition engine assigning said speech command to a recognition grammar, 
said speech command and said recognition grammar corresponding to each said instruction set; 

transmitting said speech command to said speech recognition engine; 

said speech recognition engine receiving said speech command and selecting the 
corresponding recognition grammar; 

said server retrieving each said instruction set corresponding to said recognition grammar; 
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said server accessing said pre-defined portion of said pre-selected web site defined by 
said instruction set and retrieving said information defined by said instruction set when the 
requested information is found in the pre-defined portion of the pre-selected web site; 

said server searching said pre-selected website when the requested information is not 
found in the pre-defined portion of the pre-selected web site; 

said speech synthesis engine converting the retrieved information from said pre-selected 
web site into an audio message; and 

said speech synthesis engine transmitting said audio message to said user. 

Claim 64 (Previously Presented): The method of claim 63 wherein the step of providing 
at least one instruction set to the server is performed by the user. 

Claim 65 (Previously Presented): The method of claim 63 wherein the step of providing 
at least one instruction set to the server comprises the steps of: 

displaying the web site on a graphical display operatively connected to the server; and 

using computer software to select the pre-defined portion of the pre-selected web site 
which contains the information to be retrieved. 

Claim 66 (Previously Presented): The method of claim 65 wherein the step of providing 
at least one instruction set to the server is performed by the user. 

Claim 67 (Previously Presented): The method of claim 63 wherein the pre-defined 
portion of the pre-selected web site being retrieved is periodically updated. 

Claim 68 (Previously Presented): The system of claim 53 wherein the named object is 
selected from the group consisting of: "weather", "forecast", "high", "low", "radar", "temp", 
"temperature", "humidity", "humidity level", "wind", "wind speed", "wind direction", 
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"pressure", "sunrise", "sunset", "time", "month", "day", "stock", "stock quote", "news", "news 
reel", "airline", "carrier", "flight", and "flight number". 
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EVIDENCE APPENDIX 

The Examiner has not identified where the claimed invention allegedly is disclosed in 

either U.S. Provisional Patent Application No. 60/263,003 or U.S. Patent Application No. 
09/464,989 to Wesemann et al. Consequently, these priority documents have not been entered 
into the record. 

Applicant submitted copies of U.S. Provisional Patent Application No. 60/263,003 and 
U.S. Patent Application No. 09/464,989 to Wesemann et al. along with the Amendment After 
Final filed May 21, 2007 and the Reply to the Advisory Action filed July 18, 2007, but the 
Examiner refused to enter these priority documents into the record. 

Accordingly, Applicant has attached copies of U.S. Provisional Patent Application No. 
60/263,003 (Exhibit 1) and U.S. Patent Application No. 09/464,989 to Wesemann et al. (Exhibit 
2) to this Appeal Brief as evidence. 
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RELATED PROCEEDINGS APPENDIX 
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EXHIBIT 1 



Choosit/User Defined Mapping 



ENABLING USERS TO CONFIGURE A PROCESS OF MAPPING SECTIONS OF BROWSER- 
ACCESSIBLE CONTENT 

INV: Dave Morton 
David Lloyd Summers 



Summary 

User Defined Mapping (UDM) provides a method of accessing browser-based data or information, such as 
Internet content, networked applications, custom data, field input and passowords, from the UDM Choosit 
browser. Choosit is the tool used to map specific, user-defined content, such as text or text links, and makes 
it available on any communications device, such as a rotary or cellular telephone, mobile PDA, or PC. 
Choosit allows users to voice enable specific sections of networked data by speaking a name assigned by 
the user. 

I. Background 

Problem 

The basic problem arises when a user wants to access browser-based data on his/her device using a voice 
interface. However, this cannot be done by existing methods because most of the content currently 
consists of formats such as images with JavaScript. This hinders the ability to access information over a 
voice interface in the same way a person might access networked content via traditional access methods, 
such as PC-based Internet browsing. For example, if a user goes to a Web site, s/he can navigate to a 
section within that Web site that contains articles that s/he wants to access. However, within those articles 
are numerous others. If the user accessed those articles using a voice browser, the voice browser would 
start reading every link and would finally gets to the stories the user actually wanted after the system had 
taken up a significant amount of time reading the smaller links. 

Talk2's Choosit Solution 

The traditional PC-based user experience of accessing data through a browser cannot be exactly recreated 
over the phone so the next best alternative is to enable the user to decide what gets read to him/her on the 
phone. Choosit is able to create a map to the location or section (not just static text) that the user selects 
and wants to access, via such methods as voice recognition or Internet-enabled devices. The difference 
between the section and the static text is important and a key component of UDM because the section is 
dynamically updated every time a user go to the site, the text is not. This allows the text to change as 
content is updated because Choosit has enabled the user to select the section or position in which the text 
would appear. 

While UDM also has the capability of mapping static text in addition to dynamic sections of data, it is 
desirable for the user to map all sections s/he would like to access through a device because if content has 
any formatting at all, a device is not going to read the formatted text very well. One example might be a 
stock portfolio on a Web site that a user would like read to him/her. Typically, the information is in a table 
and the end user wants the system to read the name of the stock and its value. The table may contain other 
information that the user does not really want, so if the user sets the site up so Choosit will select the name 
and value, the system will actually offer only the information that had been mapped. In addition, UDM 
will enable the mapped information - in this case, the value of the stock - to be updated dynamically so the 
user always receives the most current value. 

UDM also differs from other voice-enabling or text-to-speech solutions such as VoiceXML. Other 
solutions require the person who designed the site to go back and redesign the site to be compatible with 
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that solution. However, this site designer may not care about the end user's needs in every case With 
UDM technology the end user actually does the mapping, which offers the user two advantages. First, it 
does not matter what the owner of the content wants the end user to access, the user can just map the 
content or data and the owner of the data does not need to be involved. In this case of Choosit technology, 
the user controls content similar to way it is possible to disable formatting such as images on the browser' 
For example, if a content provider writes in VoiceXML, s/he can push content that s/he wants the end user 
to hear or see. The second advantage is that the end user has the ability to choose what s/he wants to hear 
or see. 



In addition to giving the end user the ability to map desired content for access over a voice interface, UDM 
also has the ability for others, besides simply the end user, to map data. For example, UDM allows a 
system administrator to map data behind a corporate firewall, such as a corporate intranet or CRM 
application, for remote access. 



II. How UDM Works - A High Level Overview 

1 . The user navigates to the Choosit Web page from the primary entry portal. 

2. The navigational tool loads in the left frame. On the right side of the page is a "working desktop" 
where the site of interest is loaded. (See Figure 1) 

3. The user is prompted to enter the desired content, such as a URL or custom application, which will fill 
the working desktop on the right side of the screen. The content on the working desktop will look like 
the original content form, although Talk2 will have stripped out all extraneous formatting such as the 
JavaScript, logos, images, etc. Essentially, Talk2 will have remapped all aspects of the content, 
including such items as forms or images, so that the content will work with "Choosit," (See Figure 2 
for a complete map of the Choosit end user experience) 

4. The user is prompted to choose a name for the content, which acts as a bookmark or main folder This 
"bookmark" is any user-chosen name or description given to the desired data that the user will access 
For example, "Sports" may be a bookmark name for a sports-related URL the user has chosen. 

5. After Choosit has loaded the page, it asks, "Is there anything here you would like to map?" If the user 
says "yes," then Choosit says "text" or "link," and the user clicks on the mouse to drag and select the 
text that s/he wants from the links. The user selects the desired text or link and is asked to select a 
"voicelink" name that will be spoken over the telephone to access the content. Choosit would say 
"Do you want to add text or links?" If the user selects "text" then no matter what is selected the 
content is going to be accessed by the user on the device as text. Often, the user may want links read 
over the voice portal - for example, temporal links may be the headlines of a story. The user selects 
this option by clicking on the mouse and holding it down to highlight the desired section The 
highlighting will follow the format rather than the shape of the text (for example, a rectangular 
region). b 

6. When the user releases the mouse, Choosit will ask, "Do you want to add another text area?" The user 
can keep adding links or text by repeating the process. When the user is finished selecting a section 
Choosit will say, "Enter a name," and the user enters the name of the voicelink. The link is loaded' 
assigned the name, and placed in the main folder (or bookmark). Once you select a name for the 
voicelink, a branching tree is formed with the original bookmark link. A bookmark may have several 
links under it. When a user selects the bookmark from his/her device, Choosit will enable the system 
to say for example, "This bookmark has the following links, A, B, C." 

7. Once the user selects the voicelink within the bookmark, the system would begin offering text at that 
point. If the voicelink contains additional links, the system will offer the links to the user so that s/he 
can choose among the options. For example, if a voicelink is a page of a Web site and contains a 
collection of articles, the system would list the article titles so the end user could select among the 
options. 

8. Once the voicelink is established, the user can start over again and add more voicelinks or links within 
the voicehnks. The system will ask if there's anything the user would like to map and if the user says 
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"no," then the links all just become active, the user can click on a link and follow that link, just like a 
normal browser. The user can go even deeper before actually mapping the content. 

Figure 1 illustrates the graphic user interface for UDM. There are three sections on the left and on the top 
there is the Choosit logo. 




Figure 1 



Call Flow Explanation ri S^ e2 



UDM Prompt 


UDM output 


Left frame 


Right frame 


Please enter the 
address (URL) of the 
site you want to map 


1. Entered URL name output below "Voice 
Enabling" 




Enter a name for this 
bookmark. This will 
be the name you 
speak to access this 
content. 


1 . Entered bookmark name output below 
"Book Mark Name:" 

2. If the required page is successful loaded, 
"Is there any information here you would 
like to map?" otherwise "Please enter the 
address (URL) of the site you want to map" 
is output. 


If the required page is 
successful loaded, page 
information is 
displayed; otherwise 
standard error message 
is output. 


Is there any 
information here you 
would like to map? 


Click "Yes", "Do you want to add text or 
links from this page?" is output. 
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Click "No", "Using the screen on the right 
follow the link to the page you want to 
map." is output. 




Do you want to add 
text or links from this 
page? 


Click "Text", "On the screen at the right, 
select an area of text by clicking and 
dragging over the select text." is output. 






Click 'Links", "On the screen at the right, 
select a link or group of links by clicking 
and dragging your mouse over the group." 
is output. 




Using the screen on 
the right follow the 
link to the page you 
want to map. 


Click the selected link, If the required page 
is successful loaded, "Is there any 
information here you would like to map?" 
is output; otherwise "Please enter the 
address (URL) of the site you want to map" 
is output. 


If the required page is 
successful loaded, page 
information is output; 
otherwise standard 
error message is 
output. 


On the screen at the 
right, select an area 
of text by clicking 
and dragging over the 
select text. 


Drag the select text, "Do you want to add 
another text area to this voicelink?" is 
output. 




On the screen at the 
right, select a link or 
group of links by 
clicking and dragging 
your mouse over the 
group. 


Drag the select links, "Are the pages these 
links lead to similar in layout?" is output 




Do you want to add 
another text area to 
this voicelink? 


Click "Yes", "On the screen at the right, 
select an area of text by clicking and 
dragging over the select text." is output. 




Click "No", "Enter a name for this 
voicelink." is output 


Are the pages these 
links lead to similar 
in layout? 


Click "Yes", "On the screen at the right, 
select an area of text by clicking and 
dragging over the select text." is output if 
load is successful; otherwise "Please enter 
the address (URL) of the site you want to 
map" is output. 


If the first select link is 
successful loaded, the 
page information is 
output; otherwise 
standard error message 
is output. 




Click "No", "Enter a name for this 
voicelink." is output. 




Enter a name for this 
voicelink. 


Entered name is added into bookmark tree. 
Click "Ok" after enter the name of the 
voicelink, "Do you want to add another 
voicelink to this bookmark?" is output. | 
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Do you want to add 
another voicelink to 
this bookmark? 



Click "Yes", "Is there any information here 
you would like to map?" is output. 



Click "No", "Saving Data..." is output. 



III. UDM Technical Specifications 



"This specification details the development of the functions and processes that are capable of accessing 
general browser-accessed content using UDM for retrieval from any device, such as a telephone PC or 
Internet-enabled PDA. ' ' 

Requirements 

UDM should be able to load any kind of data, such as networked data or content accessed from a browser. 
However, UDM cannot map the following types of data: 

• Pages information that is dependent on JavaScript (VBScript) programs. 

• SSL (Secure Sockets Layer) pages whose URL generally begin with "https://..." 

• Any page that has an extension of ,ps, .pdf, .cfin, or .exe. 

• Pages with enabled flash objects. 
;n • Video player pages. 

r-y For those pages that UDM cannot load, UDM will display a standard error message to users, while the 
fj\ unloaded page and identified real error message are saved into a database table for TTtalk2 engineers who 
; y work with problem determination. 

H 

Design 

;7; See Figures 4 and 5 for a graphic depiction. 

* The user goes to the primary access portal and chooses the information s/he would like to map. Talk2 then 
^ goes to that site and pulls the requested content and returns the information to the user - Talk2 basically 
f t becomes a proxy. When it returns the text to the user Choosit also re-maps the content site and reroutes the 
location through the proxy rather than linking directly to the location. (See Figure 3 for a diagram of the 
UDM architecture.) The remapped location, such as a Web site, is still functional, but it has got a Talk2 
^ wrapper around it. UDM actually parks the content and remaps it when Talk2 is acting as a proxy. 
?* Through this process, UDM is storing those offsets (collection of selected characters) locally on the user's 
box and doesn't actually save anything until the end. The system waits until the user gets to the end and it 
says "Do you want to add another voicelink to this bookmark," and the user says "no." Then, UDM 
submits all of the user's mapping to the server and saves it to the database. When the user is done' UDM 
sends it again to the Talk2 Web server and this time the Talk2 Web server writes it to work with database 
This user-defined map is stored in the Talk2 database. (See Figure 6 for details about the database.) For 
example, a user would like to access Choosit-defined information by telephone. The user would dial into 
the telephone node and this database would get an identifier for this user and find the map associated with 
that person. This map is not just limited to the end-user, however, because there could also be a default 
map stored in the database regardless of who the user is. 



Prompt instructions 



Response, 



5 



Display requested information Send request 



Browser 




Data 
base 



' r-T ' Store mapped information 

pass parameters send requested information 



Javabean: 
udmbean 



connect to server 
request information 



er A ] 

>n s 

Internet 
Server 



> response, 

send back information 



UDM actually saves the original content site as a collection of offsets in that site. UDM translates those 
offsets into the new page using what is called the difference function. This difference function is how 
UDM picks up changes in the selected content For example, if a user would like to access Choosit Web 
content through a voice portal, the user calls into the telephony node, and the telephony node talks to the 
database and gets this original map of the content. The telephony node then talks to the site and gets the 
new page and determines the difference between the originally chosen content and the changes that may 
have been made to that URL and matched them up on a byte-by-byte basis. 

If the chosen content location makes some changes to the selected data, the difference function allows the 
selected section to be dynamically updated. For example, if the function originally says, "I am a frog" and 
you want it to read the word "a frog." UDM creates a start pointer and an end pointer based on the start of 
the section and end of the section the user selects. The location changes the content so it says, "You are a 
funky frog." The difference function is going to find the best fit between these disparate phrases. In this 
case, the difference function will read through the text and will find a space followed by an "a." The 
difference function will determine there is probably a match between the spaces and words (in this case, 
"frog" would match). Based on the matches, the difference function will determine what characters were 
insertions and deletions. The difference function will then determine that based on the location of the 
deletion, the offset would move. In this example, UDM will start at the relocated offset and would stop, 
based on the original "end pointer". The system will say "a funky frog" instead of "a frog." With HTML 
code, this difference function works much better than this example demonstrates because HTML code 
actually has structure described in it and so the algorithm automatically selects the best way to match this 
content. If all of the text is the same, except slight changes, then UDM uses text. But if all other factors 
are different except for the structure, then UDM uses the structure. 
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Specification Detail 
UDM High Level Design 



Figure 4 



Open a window to provide UDM Choosk browser at talk2 server; prompt a user to input URL I 



Get the input URL; prompt user to input bookmark name for this site and get it; Initinize arrays for I 

5F 1 



<^J^ss URL to udmbean~~~^> 




Go to standard error message 
& page(errorPage.jsp): store the URL and 
=. the error message into UDMLoadError 
table at CU STOM database; display 
standard error message. 



Deal page source file so Choosit would successfully display the 
page: remove all script code and flash object; check whether need 
redirect to another page; find full path for every link and image 
source; remap server for form; check whether page contains frameset 



Display dealed source file information; prompt the user 
either go to another page or map something on this 



Get the selected links; find the start and end offset for 

every selected link on the page; prompt the user 
whether he want the page links lead to similar in layout 



Get the selected text; find the start and end 
offset of the selected text on the page; prompt 
the user whether he want to add another text. 



Prompt the user to input a 
name for the voicelink. 



Prompt the user whether he wants to 
add another voicelink to the bookmark. 



Save book-marked information 



database 



UDM Functionalities Call 



Figure 5 



Open UDM Choosit Browser 




Standard 
Error 

message 

on the 
right frame 
of Choosit 

browser 



Save URL, and 
identified real error 
message into 

UDMLoadError 
Database 

available 



Is there anything here 
you would like to map? 




Display 
required page 
information 
on the right 
frame of 
Choosit 
browser 



Using the screen on the right follow 
the link to the page you want to map 



^ Links ^ Cjjjjj 
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On the screen at the right, select 

some text by clicking and 
dragging your mouse over the 



On the screen at the right, select a 
group of links by clicking and 
dragging your mouse over the group 




I^atahase Height lev el Design 




Saving ^> 



Choosit 
browser 



pass parameter 



openCormection(String db) 
setBookMark(BookMark bookmark) 
addBookMark(BookMark bookmark) 
getBookMark(BookMark bookmark) 
AddBookMarkPages(int pageid,String text) 
delBookMark(BookMark bookmark) 
addErrorMessage(UDMLoadError errorTable) 
getUDMLoadError(UDMLoadError 
errorTable) 




Database schema 

The following database tables are located in CUSTOM database. 



BookMark table 



Field Name 


Type 


Description 


BookMarkID (key) 


Number 


ID assigned by UDM for this bookmark 


AccountID 


Number 


This is the account ID for the owner of this 
bookmark 


FolderType 


Number 


Option (value could be 1 to 9 that indicate 
which dial number is assigned to the 
bookmark) 


BookMarkName 


Text (50) 


This will be the name for user to speak to 
access this bookmark (content) 



BookMarkLink table 



Field Name 


Type 


Description 


BookMarkLinkID (key) 


Number 


ID assigned for each bookmark link 


BookMarkID 


Number 


ID assigned for this bookmark 
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BookMarkLinkName 


Text (50) 


a name for this marked voicelink. 


BookMarkLinkURL 


Text (200) 


URL of the web site which book marked 
some links 


BookMarkLinkStartOfs 


Number 


Start offset of the book marked link on 
the web site 


BookMarkLinkEndOfs 


Number 


End offset of the book marked link on the 






web site 


BookMarkLinkOrigTextID 


Number 


ID assigned for the book marked link 


BookMarkLinkTextOrigTextID 


Number 


ID assigned for the book marked text 


BookMarkLinklsLinkGroup 


Number 


Flag whether this bookmark is a group of 
links 


BookMarkLinklsDynaraic 


Number 


Flag whether this book marked pagels 
dynamic 


BookMarkLinkPostData 


Text (100) 


Post data of this book marked link 



BookMarkText table 



Field Name 


Type 


Description 


BookMarkLinkTextTD 
(key) 


Number 


ID assigned for the mapped text 


BookMarkLinkID 


Number 


ID assigned for the mapped link 


BookMarkLinkTextStartOfs 


Number 


Start offset of the mapped text in the Web page 


BookMarkLinkTextEndOfs 


Number 


End offset of the mapped text in the Web page 


BookMarkPages table 


Field Name 


Type 


Description 


BookMarkPagesID (key) 


Number 


ID assigned for this web page. 


BookMarkLargeText 


Memo 


Entire web page source file that the user maps 
some text or links from it. 


UDMLoadError table 


Field Name 


Type 


Description 


UDMLoadErrorlD (key) 


Number 


ID assigned for the error 


URL 


Text (250) 


URL of the unloaded Web page 


errorMessage 


Text (150) 


Identified real error (reason of why unable 
load the URL page) 
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and 

RICHARD T. NEWTON 
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VOICE INTERFACE FOR ELECTRONIC DOCUMENTS 



1 BACKGROUND OF THE INVENTION 

2 1. The Field of the Invention 

3 The present invention relates to methods, systems, and computer program products 

4 for accessing electronic documents. More specifically, the present invention relates to 

5 methods, systems, and computer program products for providing a voice interface to 

6 electronic documents. 



2. The Prior State of the Art 

As computers have become ubiquitous in our day-to-day activities, the advantages of 
storing information electronically have steadily increased. One of the primary advantages of 
electronically stored information is its inherent versatility. For example, editing and 
exchanging electronic information is greatly simplified as compared to editing and 
exchanging documents stored in paper form only. Furthermore, any advantage attributable 
to having a physical document is retained in electronic storage because a "hard copy" of an 
electronic document may be readily produced from the electronic version. 

Another significant advantage of electronically stored documents is that of providing 
enhanced access to information. Over the past few years, the improved access offered by 
electronic documents has become so important that many organizations expend substantial 
resources in scanning paper documents to store them electronically. Routine facsimile 
transmission further exemplifies the value of electronic access to documents. Arguably, it is 
access to information that fuels what many refer to as the Information Age. 

Today, perhaps the most prominent example of access to electronically stored 
information is the Internet. Literally millions of people depend on the Internet for email, 
banking, investing, shopping, news, entertainment, and social interaction.. Not too many 
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years ago, sharing information over the Internet was principally the domain of academicians 
and scientists. For members of the general public, the cryptic nature of access tools and the 
essentially prohibitive computer hardware requirements meant virtual anonymity for the 
Internet. However, the advent of hypertext navigation and the World Wide Web ("Web"), 
in conjunction with modestly priced and increasingly powerful personal computers, has 
propelled the Internet to the forefront of public attention and has made the Internet an almost 
indispensable source of information. 

Likewise, use of early cellular telephone technology was also limited. Initially, 
problems included providing coverage beyond major metropolitan areas, the expense and 
size of cellular telephones, and the expense of airtime. As a result, cellular telephones were 
used mostly for vital business concerns rather than for personal matters. Over the past few 
years, however, the cellular industry has solved, to one degree or another, most of the 
problems that inhibited cellular's general acceptance. Today, cellular telephone use has 
dramatically increased and, for many people, is the primary means of communicating with 
others. 

Increasing dependence on cellular telephones as a primary means of communication 
together with increasing dependence on the Internet as a source of information presents an 
unfortunate problem: a primary means of communication, the cellular telephone, does not 
interface well with a vital source of information, the Internet. The problem is compounded 
in that the hypertext navigation of the Web is visually oriented, making a computer with a 
relatively large screen an obvious choice for access, yet the size of cellular telephones is 
much more conducive to convenient portability. Frequently cellular telephones are clipped 
to belts or placed in pockets or purses; portable computers require their own case and a free 
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hand to carry. Moreover, public telephones are available to those who do not carry cellular 
telephones, whereas public computers have a minimal presence at best. 

Although the prior art includes some attempts to solve the problem of accessing 
electronic documents by voice, none of the prior art teachings offer the comprehensive 
solution provided by the present invention. Specifically, Figures 1 and 2 show the prior art's 
approaches to accessing Internet documents, approaches that have proven to be generally 
inadequate in many ways. The approach designated generally at 100 illustrates a Source 110 
of electronic content that is accessible through Telephone 120. The content in Source 1 10 is 
written in a markup language specifically designed for telephone access. 

Using Motorola's Voice extensible Markup Language ("VoxML"), the information 
includes explicit elements or tags for enabling voice interaction. However, requiring 
explicit voice elements presents a serious drawback: it provides no means for accessing 
nt that does not include the VoxML's voice elements. Thus, VoxML provides no 
access to the wealth of content already available on the Web, written mostly in HyperText 
Markup Language ("HTML"). In other words, to provide full Web access, the entire content 
>f the Web would need to be rewritten to include VoxML's explicit voice tags. 

Moreover, VoxML's facilities for authoring voice content do not provide for using a 
common source to generate both audio and visual interfaces. Therefore, even if a single 
document contains both visual and audio elements, the elements must be maintained 
separately; any changes to one must be replicated in the other. 

Figure 2 shows another approach to the problem, designated as 200, that has proven 
to be generally inadequate. HTML Source 210, representing existing Web content, can be 
accessed through one of two interfaces. First, as is well known in the art, Visual Browser 
220 provides a visual interface for Monitor 230. Second, Static Translation 240 provides an 

- Pa 8 e4 - t I Dock* No. 14999.3 



audio interface for Telephone 250. Static Translation 240 is a copy of at least a portion of 
HTML Source 210 that has been manually altered to include audio elements. Someone 
examines HTML Source 210, creates a corresponding audio interface, and then stores the 
audio interface in Static Translation 240. A user who is interested in accessing HTML 
Source 210 through telephone 250 interacts with the audio interface provided by Static 
Translation 240. 

The solution of Figure 2 has the advantage of providing an audio interface without 
obligating HTML content providers (e.g., providers of HTML Source 210) with the 
responsibility of maintaining an audio interface. However, this approach imposes new 
problems that may be nearly equal to the one it proposes to solve. Like the approach in 
Figure 1, a significant amount of work must be devoted to identifying HTML content of 
interest and then modifying that. Once the content has been initially modified, each time 
HTML Source 210 changes, corresponding changes must be made to the Static Translation 
240. Naturally, some delay will occur between the time HTML Source 210 changes and the 
corresponding modifications are made to Static Translation 240. For content that changes 
frequently, such as information regarding financial markets, frequent and constant updating 
is a significant burden. Moreover, because of the incredible amount of HTML content 
available on the Web, only a small portion could be modified to include an audio interface 
and placed in Static Translation 240, leaving vast Web content completely inaccessible to 
Telephone 250. 

One area that may be particularly well-served by telephone access is the personal 
home page market, as it is becoming increasingly popular for content providers, such as 
Yahoo!, to offer personal Web home pages. These personal pages allow a user to select 
from a variety of content that is placed on a single Web page. For example, a user may 



1 chose to have current data regarding various financial markets, weather, sports stories, 

2 headlines, technology, calendaring, contacts,_entertainment, travel, reference, etc., appear on 

3 a personal home page. By providing a single, convenient source of diverse information, 

4 these personal home pages are highly attractive. 

5 There is no end in sight for the increasing growth of the Internet nor is it likely that 

6 the Internet's expanding importance as a source of information will diminish any time soon. 

7 Considering the corresponding growth in cellular telephone use and the cellular telephone's 

8 convenient size, providing cellular access to the Internet in particular and electronic content 

9 in general would be a great benefit. Furthermore, public telephones also could provide 

10 beneficial Internet access for those who do not carry cellular telephones. However, the prior 

1 1 art lacks effective methods, systems, and computer program products for providing voice or 

12 audio interfaces to electronic content. 
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SUMMARY OF THE INVENTION 

The problems in the prior state of the art have been successfully overcome by the 
present invention, which is directed to methods, systems, and computer program products 
for providing a voice interface to electronic documents. The present invention allows for 
access to existing electronic content without requiring any modification to the content 
source. Furthermore, the present invention allows for a common content source to 
incorporate both a visual and audio interface, without including separate markups for each 
interface, making the content source more easily maintained. Although embodiments of the 
present invention are described as applied to Web pages in an Internet context, the invention 
is not limited to any particular format of electronic information or any particular network 
typically used for accessing electronic content. 

In one preferred implementation, the present invention works with content that 
operates as an index to additional content, such as is typical with personal home pages. The 
present invention takes the content of a personal home page and creates a hierarchy of 
categories that are presented to a client. There is no requirement that the client is necessarily 
person. For example, the client may be an intervening service needing an audio interface to 
electronic documents. The present invention generates an audio representation of the 
available categories and allows the client to select one. Navigating through the hierarchy, the 
client may eventually reach the bottom hierarchy level, with links pointing to content that 
includes text mixed with links. At this point, the present invention reports the number of 
links, and provides an audio representation of the text. 

Because creating categories requires some knowledge of the layout for personal 
home pages, Web content in general will not be mapped into various categories. For 
unmapped content, the present invention operates as described above with respect to text 
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mixed with links, by reporting the number of links on a page and providing an audio 
representation of the page's text. Alternatively, a client may choose to hear an audio 
representation that only includes links. In response, the client may select a link of interest to 
follow. The present invention also provides a variety of global commands that are available 
to assist navigation. 

The foregoing methods, systems, and computer program products provide significant 
advantages over the prior art. Because the present invention provides an audio interface 
without requiring any modification to existing content, the telephone access will be readily 
available to the vast information available electronically. Moreover, the present invention 
also provides for organizing certain content by mapping links and text to a hierarchy of 
categories to aid navigation. 

These and other objects, features, and advantages of the present invention will 
become more fully apparent from the following description and appended claims, or may be 
learned by practicing the invention as set forth below. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

A more extensive description of the present invention, including the above-recited 
features, advantages, and objects, will be rendered with reference to the specific 
embodiments that are illustrated in the appended drawings. Because these drawings depict 
only exemplary embodiments, the drawings should not be construed as imposing any 
limitation on the present invention's scope. As such, the present invention will be described 
and explained with additional specificity and detail through use of the accompanying 
drawings in which: 

Figure 1 is a block diagram showing a prior art solution for providing a voice 
interface to electronic content; 

Figure 2 is a block diagram showing another prior art solution for providing a voice 
interface to electronic content; 

Figure 3 is a block diagram illustrating the relationship of the present invention to 
other components used in accessing electronic content; 

Figure 4 is a block diagram showing increased detail of the components that make up 
the present invention; 

Figure 5 is a flow chart illustrating a preferred embodiment of the present invention 
that includes the use of mapped categories; 

Figure 6 is an example of electronic content that is used to describe the operation of 
the embodiment illustrated in Figure 5; 

Figure 7 shows the portfolios portion of the content from Figure 6 in greater detail; 
Figure 8 shows the weather portion of the content from Figure 6 in greater detail; 
Figure 9 shows the headlines portion of the content from Figure 6 in greater detail; 



Figure 10 illustrates the hierarchy generated by the present invention for the content 
shown in Figures 6-9; 

Figure 1 1 is a flow chart illustrating a* preferred embodiment of the present invention 
that does not include the use of mapped categories; and 

Figure 12 is an example of electronic content that is used to describe the operation of 
the preferred embodiment illustrated in Figure 1 1 . 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The invention is described below with reference to drawings. These drawings 
illustrate certain details of specific embodiments that implement the systems, methods, and 
computer program products of the present invention. However, describing the invention with 
drawings should not be construed as imposing, on the invention, any limitations that may be 
present in the drawings. For example, the embodiments that follow describe the present 
invention in the context of Web pages usually accessed over the Internet. Nevertheless, the 
scope of the present invention is not limited to electronic content formatted as Web pages 
nor is it limited to content that is ordinarily accessed through the Internet. 

The present invention relates to methods, systems, and computer program products 
for providing an audio interface to electronic content. Two embodiments are described 
below. Each embodiment is a significant advance over the prior art because no modification 
of the content's source is required. The first embodiment is most useful for content that is 
arranged as a hierarchical index, with broad topic indices leading to more specific topic 
indices and eventually to individual documents discussing a particular subject. The present 
invention creates a hierarchy of categories and indices. A corresponding audio 
representation allows a client to navigate through the content, where the client need not be a 
person. For example, the present invention could be accessible to other services needing a 
voice interface to electronic content. Upon reaching the bottom level in the index hierarchy, 
selection of a link leads to specific documents. 

Reaching specific documents introduces the operation of the second embodiment. 
Here, the present invention identifies the number of links and provides the user with an 
audio representation of the document text. A client may also choose to hear the links to 
navigate among various documents. Depending on the initial page identified by a client, the 
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present invention may begin operating according to either of these two embodiments. Each 
embodiment includes the benefits of providing an audio interface to dynamic Web content 
without requiring providers to modify their documents. 

The embodiments of the present invention may comprise a special purpose or 
general-purpose computer comprising various computer hardware. Other embodiments 
within the scope of the present invention also include computer-readable media having 
computer-executable instructions or data structures stored thereon. Such computer-readable 
media can be any available media that can be accessed by a general-purpose or special- 
purpose computer. By way of example, and not limitation, such computer-readable media 
can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk 
storage or other magnetic storage devices, or any other medium which can be used to store 
the desired executable instructions or data* structures and which can be accessed by a 
general-purpose or special-purpose computer. 

When information is transferred or provided over a network or other 
communications connection to a computer, the computer properly views the connection as a 
computer-readable medium. Thus, such a connection is also properly termed a computer- 
readable medium. Combinations of the above should also be included within the scope of 
computer-readable media. Computer-executable instructions comprise, for example, 
instructions and data which cause a general-purpose computer, special-purpose computer, or 
special-purpose processing device to perform a certain function or group of functions. The 
computer-executable instructions and associated data structures represent an example of 
program code means for executing the steps of the invention disclosed herein. 

The invention will be described in the general context of computer-executable 
such as program modules, being executed by a computer. Generally, program 
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modules include routines, programs, objects, components, data structures, or the like that 
perform particular tasks or implement particular abstract data types. Moreover, those skilled 
in the art will appreciate that the invention may be practiced with other computer system 
configurations, including hand-held devices, multi-processor systems, microprocessor-based 
or programmable consumer electronics, network PCs, minicomputers, mainframe 
computers, and the like. The invention may also be practiced in distributed computing 
environments where tasks are performed by remote processing devices that are linked 
through a communications network. In a distributed computing environment, program 
modules may be located in both local and remote memory storage devices. 

Turning now to Figure 3, an environment, including the present invention, for 
accessing electronic content is referenced generally as 300. HTML Source 310 is an 
example of electronic content that is common to the Web. However, the invention imposes 
no particular requirement on the format of the content's source or on how the content 
typically is accessed. Visual Browser 320 accesses HTML Source 310 and provides a visual 
representation for Monitor 330. Visual browsers, such as Microsoft's Internet Explorer and 
Netscape's Navigator are both well known in the art. Voice Browser 340 provides an audio 
interface to HTML Source 310 that is suitable for use by Telephone 350. Alternatively, 
Voice Browser 340 could be used in conjunction with Visual Browser 320 to provide 
simultaneous visual and audio interfaces. Similarly, Visual Browser 320 could also be 
specialized to generate content that would be suitable for the limited space of a telephone 
display. Then, Visual Browser 320 and Voice Browser 340 could be used simultaneously 
through Telephone 350. 

Figure 4 shows some of the basic components that make up Voice Browser 340. In 
addition to the following relatively brief treatment, the operation of these basic components 
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1 will be described in greater detail with respect to the flow chart of Figures 5. Line/Call 

2 Manager 410 is responsible for establishing and maintaining telephone connections. Text to 

3 Speech 420 converts the text it receives to speech that can be communicated to a client and 

4 is an example of processor means for generating an audio representation of electronic 

5 content. Text to Speech 420 may also include some prerecorded speech. For example, 

6 prerecorded speech could be used for frequently used words, links, text or prompts. Modules 
1 for implementing both Line/Call Manager 410 and Text to Speech 420 are well known in 

8 their respective arts. Document Parsing and Audio Layout 430 receives electronic content 

9 and identifies any text and links included within the electronic content and is an example of 

10 processor means for parsing electronic documents. (Links are content elements that lead to 

11 other locations in the same document or to other documents entirely. HTML links, for 

12 example, create locations within a document's visual representation that may be selected to 

13 further explore the link's subject, such a defining a word or leading to related material.) The 

14 audio layout portion may organize certain content into a hierarchy as an aid to navigation 

15 and is an example of processor means for mapping any text and links identified into one or 

6 more categories. 

7 Speech Recognition 440 interprets the audio or voice data received from a client so 
S that Command Processing 450 can execute the client's request. Speech Recognition 440 is 
9 an example of processor means for receiving a spoken instruction from a client. Modules for 

0 implementing Speech Recognition 440 are-well known in the respective art. Command 

1 Processing 450 may also perform various general control functions and coordinate the 

2 operation of other components. Document Retrieval Protocols 460 request and receive the 

3 electronic content of interest and are examples of processor means for obtaining electronic 

4 documents and for following links. These Document Retrieval Protocols 460 are also well 

- Page 14 - j , Docket No. 14999.3 

r 



%8<$P i 



known in the art of accessing electronic content, especially in the context of HTML 
documents. 

As described in Figures 3 and 4, Voice Browser 340 provides an audio interface 
without imposing the limitations found in prior art solutions. Specifically, Voice Browser 
340 does not require content providers to modify their documents to support a voice 
interface. Therefore, the dynamic content of the Web is available to Voice Brower 340 at the 
same instant it is available to Visual Brower 320. How Voice Browser 340 operates to create 
an audio interface is described more fully with reference to Figure 5. 

All acts shown in the flow chart of Figure 5 will be described by using the document 
shown generally in Figure 6, and more specifically in Figures 7-9. Because each figure 
number is incorporated into individual references, i.e., reference 650 appears in Figure 6 and 
reference 940 appears in Figure 9, the specific figure number may be inferred and therefore 
may not be explicitly identified in the discussion that follows. It should also be noted that 
while the steps of Figure 5 are shown sequentially, there is no requirement that one step be 
completed prior to the next step beginning. For example, the prompts can be interrupted or 
anticipated by making a selection before the prompt finishes or before it even begins. Figure 
6 is an example of content that provides hierarchical indices leading to more textually 
oriented material and is suitable for enhanced mapping. 

In step 5 1 0, a particular document is identified or selected. For example, Text to 
Speech 420 may prompt the client to select or request a desired source of information. 
Options include unified messaging, home page, favorites, etc. Prompts for unified 
messaging, home page, favorites, etc., are examples prompts that may be prerecorded and 
included in Text to Speech 4^0. In response, the client selects the personal home page 
shown in Figure 6. The present invention can also include a variety of global spoken 
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navigation commands, such as fast forward, rewind, cancel, forward, back, home, links, fax, 
telephone, and email. Fax, telephone, and email are instructions to fax, telephone (voice 
mail), or email the current document's contents, or some portion thereof, in audio and/or 
visual form based on what is appropriate for the particular instruction given, to someone 
selected from the client's contact list; the other terms retain their ordinary meaning. For 
example, an instruction to fax would send a visual representation of at least a portion of the 
document's contents to the fax recipient. Links is a request to hear a page's links only rather 
than its text. 

Next, in step 520, Document Retrieval Protocols 460 retrieve or obtain the 
document. No particular protocols are imposed according to the present invention. For 
example, the document may be stored locally, stored on a local area network, stored on a 
private wide area network, or stored on the Internet. The document shown in Figure 6 is 
retrieved from the Internet. Having obtained the requested document, in step 530 Document 
Parsing & Audio Layout 430 next parses the content to identify any title, any text, any links, 
and any link names included within the document. A link name is simply the text that forms 
the link. For example, "Weather" is the link name of Weather category 810. 

Parsing the retrieved document to identify title, text, links, and link names that may 
be present illustrates how an audio interface may be provided without requiring changes to 
the document source. In conjunction with the other aspects of the present invention, this 
allows immediate audio access to dynamic visual content that otherwise would be 
unavailable in the prior art. Once parsed, in step 540 the text and links included within the 
document are mapped to various categories. 

Figure 6 identifies the categories present in the selected document and also shows 
some portions of the document that are filtered out and ignored. Top Banner 610 and 



Bottom Banner d60 include a 



lety of images and other content that is not particularly 



suitable for voice interaction. However, the enhanced mapping identifies three categories of 
information stored on the page, Portfolios 630, Weather 640, and Headlines 650. Each of the 
categories may also include content that is ignored. For example, Graphic 840 (see Figure 8) 
is eliminated because there is no speech analog, although alternate information provided 
within the image tag, such as the text of the "alt" attribute, could be used. Search Fields and 
Instructions 770 and 850 (see Figures 7 & 8) are eliminated because it is impractical to enter 
this type of data by speaking into a telephone. 

There are a variety of ways to identify the page content that should be mapped. For 
example, it may be possible to use HTML tags, including attributes, as an indication of 
various categories. The enhanced mapping feature of the present invention for My Yahoo! 
pages looks for a tag with a particular background color attribute. Other mappings may use 
other HTML tags and/or tag attributes to identify categories. While enhanced mapping 
beyond the default mapping provided by parsing text and links requires some degree of 
customization, a single mapping can be used for all corresponding pages provided by a site. 
Thus, a single My Yahoo! enhanced mapping provides enhanced mapping for all My 
Yahoo! pages. 

Figure 10 shows the hierarchy created by enhanced mapping of the document shown 
in Figure 6. The Categories 1010 include Portfolios 710, Weather 810, and my Front Page 
Headlines 910. The First-level of Links 1020 includes Quotes 720 within the Portfolios 710 
category, Salt Lake City, UT 820 within the Weather 810 category, and Top Stories from 
Reuters 920, Tech News from News.com 930, and Top Sport Stories from AP 940 within 
the my Front Page Headlines 910 category. The Second-level of Links 1030 includes DJIA 
730 and NASDAQ 750 within the Quotes 720 first-level of Portfolios 710 and the individual 



story headlines 922-926, 932-936, 942-946 within the first-level links Top Stories from 
Reuters 920, Tech News from News.com 930, and Top Sport Stone; from AP 940 all within 
the My Front Page Headlines 910 category. The Text of Stories 1040 are documents that are 
produced by selecting any of the Second-level Links 1030. 

In step 550, Text to Speech 420 generates the audio representation that corresponds 
to the document. It is not necessary that all of the audio representation be generated at one 
time. For example, a portion of the audio may be generated and communicated to the client 
while another portion is being generated. The audio may also be generated on demand as 
each level in the mapped hierarchy is accessed. 

Next, in steps 560-590, Text to Speech 420 prompts the client to make various 
selections from categories 1010, First-level Links 1020, and Second-level Links 1030 to 
reach Text of Stories 1040. Again, some of these prompts may be prerecorded. Because 
each of the categories shown in Figures 7-9 includes options that may not be relevant to or 
available in other categories, steps 560-590 will be discussed separately for Figure 7, Figure 
and Figure 9. Thus, steps 560-590 represent all possible choices. For certain documents, 
some of the steps may not be required. The foregoing description presumes that the 
selections made in steps 560-580 do not result in a document that is mapped. 

Assuming that a client chooses Portfolios 710 in response to the category selection 
prompt in step 560, the following will occur? Because Portfolios 710 includes only a single 
first-level link, Quotes 710, prompting in step 570 is skipped, but the text of Quotes 710 
played, and the client will be prompted to select a second-level link, either DJIA 730 or 
NASDAQ 750 (i.e., "Quotes, please choose from DJIA or NASDAQ"). In step 590, 
choosing DJIA 730 will play audio of Text 740 and choosing NASDAQ 750 will play audio 
of Text 760. However, DJIA 730 and NASDAQ 750 are also links. Although choosing the 



Links global command would not alter the choices offered, it would alter the action taken by 
making a selection. In this case choosing DJIA 730 or NASDAQ 750 would follow the 
respective links rather than playing the audio representation of Text 740 or Text 760. 

Selecting Weather 810 at step 560 similarly leads to skipped steps. However, in this 
case, both steps 570 and steps 580 are skipped because Salt Lake City, UT 820 is the only 
first-level link and there are no second-level" links. Therefore, selecting Weather 810 will 
result in the audio representation of Text 830 being played (i.e., "Salt Lake City, UT, 49 to 
82 F") at step 590. A Links command could also be issued here to identify Salt Lake City, 
UT 820, but the link would only be followed if the client explicitly selected it. 

In contrast, selecting My Front Page Headlines 910 at step 560 does not result in any 
skipped steps. In step 570, the client will be prompted to select from the first-level links Top 
Stories from Reuters 920, Tech News from News.com 930, and Top Sports Stories from AP 
940. Selecting any of these first-level links in step 570 will result in step 580 prompting for 
the stories associated with the first-level link. For example, selecting Top Sport Stories from 
AP 940 in step 570 will lead to step 580 prompting the client to select from NL Playoffs 
Notebook 942, NFL Roundup 944, and America's Cup Enters Third Day 946. In step 590, 
an audio representation of the document text corresponding to .the selection made in step 580 
will be played to the client. 

Portfolios 710, Weather 810, and My Front Page Headlines 910 present a large 
amount of information to the client. As the client moves from one category to another, each 
category presents an increasing number of links or options. In a visual environment, it is a 
relatively simple matter for the client to scan a page and remember the links or options that 
currently available. However, in an audio representation, it is significantly more 
difficult to keep the links and options of one page separate from the links and options of 




another page. Therefore, one aspect of the present invention accumulates all links and 
options from certain pages that are visited and makes the accumulated links and options of a 
previously visited page available in a subsequent page. 

Accumulation is desirable because "pages" are a visual motif that does not 
ily carry over into an audio representation. Particularly in a personal home page 
environment, a client may view the personal home page as simply a monolithic source of 
information. Someone familiar with the available content who is moving between various 
levels in the hierarchy, may find an explicit requirement of returning to a particular page, for 
the sole purpose of selecting a link or other option from that page, cumbersome or even 
' annoying. Therefore, accumulation preserves the organizational benefits of hierarchical 
organization-the client continues to be informed regarding the content of a particular 
page-without limiting the availability of links to only those present on the particular page. 

For example, selecting the NL Playoffs Notebook 942 link otTop Sport Stories from 
AP 940 found in the category My Front Page Headlines 910, will lead to the NL Playoffs 
Notebook document. That document contains both text and links that are available to the 
client. In a typical visual browser, if the client next wanted to choose category Weather 810, 
the user would need to return to Web Page 600 first. However, the present invention, by 
accumulating links, would allow the client to select Weather 810 from the NL Playoffs 
Notebook document since Web Page 600 had been previously visited. In a preferred 
embodiment, accumulation is limited to certain predetermined Web content that would 
benefit from the feature, such as personal home pages. 

In contrast to Fi/re 5, Figure 11 is a flow chart illustrating the operation of a 
preferred embodimen/f the present invention that provides only the default mapping of 
separating text an/links. In step 1 1 10, a client selects the option of starting with a list of 
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favorite Internet sites. Text to Speech 420 prompts the user to select one of the sites listed, 
step 1120. As bfcfore, in step 1130 the document in retrieved using the protocols that are 
appropriate giWn the document's location. Again, as before, the content is parsed in step 
1 1 40 to identify any title, any links, any link names, and any text included in the document. 

Figure 12 is an example of Electronic Content 1200 that is useful in describing the 
embodiment disclosed in Figure 11. Parsing step 1 140 identifies title 1240 (HTML titles are 
indicated by explicit tags), links and corresponding link names 1210, 1220, and 1230, as 
well as the document text. With these various elements identified, ui audio representation of 
Electronic Content 1200 is generated in step 1150. Once generated, the document title is 
reported in step 1 1 60 and is followed by reporting the number of links in step 1 1 70. 

Just as with the description of Figure 5, parsing the retrieved document to identify 
any title, any text, any links, and any corresponding link names that may be present shows 
how an audio interface may be provided without requiring changes to the document source. 
Together with the other aspect of the present invention, this provides immediate audio 
access to dynamic, visually-oriented, content that otherwise would be inaccessible to the 
prior art. 

In the case of the Electronic Content 1200, the present invention reports the 
document title as Guide to Filing a Utility Patent Application 1240. There are three links, 
named U.S. Patent and Trademark Office 1210, www.uspto.gov 1220, and Patent and 
Trademark Depository Library 1230. Then, in step 1180, an audio representation of the 
electronic document's text is played for or communicated to the user. In the case of 
Electronic Content 1200, this text includes everything except the title 1240. The client may 
also choose the global Links command, to hear an audio representation of the three links, 
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1210, 1220, and 1230. By choosing a link, the client instructs the present invention to 
follow the link, as in step 1 120, beginning audio interface process anew at step 1 130. 

The present invention may be embodied in other forms without departing from its 
spirit or essential charactenstics. As properly understood, the preceding description of 
specific embodiments is illustrative only and in no way restrictive. For example, using Web 
s accessible over the Internet to describe the present invention does not limit the 
invention to any specific format of electronic content or any particular means of accessing 
electronic content. The scope of the invention is, therefore, indicated by the appended 
claims as follows. 

What is claimed and desired to be secured by United States Letters Patent is: 
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frra system thitt includes an information service~arid~iffl input/output device 
for interacting with- the information service, a method of providing an audio interface for 
accessing electronic documents, the method comprising the acts of: 

obtaining an electronic document; 

parsing thej electronic document to identify any text and any links included in 
the electronic document; and 

generating an audio representation of at least a portion of the parsed 
electronic docunfent, the audio representation being communicated to a client. 

2. A method as recited in claim 1 further comprising the acts of: 

obtaining at least two electronic documents, one of the at least two electronic 
documents being designated as a prior electronic document and one of the at least 
two electronic documents being designated as a subsequent electronic document; 

parsing the at least two electronic documents to identify any text and any 
links included in the at least two electronic documents; and 

making any links identified in the prior electronic document available in the 
subsequent electronic document. 

3. A method as rfccked'V claim 1 further comprising the act of mapping said 
any text and said any links D4cmded4n the electronic document to one or more categories. 

4. A method /as recited in claim 3 further comprising the acts of generating an 
audio representation of tie one or more categories. 
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yB) A method as recited in claim /( further comprising the act of prompting the 
client to select one of the one or more categories. 

6. A method as recited in claim 3 wherein the act of mapping includes creating 
ahierarchy of said any text and said any links. 

Jl . A method as recited in claim 1 further comprising the act of receiving an 



A method as recited in claim j wherein the instruction received from the 
client is at least one of an instruction to email, fax, or voice mail at least a portion of the 
electronic document to a third party selected from the client's contact list. 



A method as recited in claim j wherein the instruction is spoken into the 



1 } 

A method as recited in claim J wherein the audio representation includes at 
least one link and wherein the instruction from the client is to select the at least one link, 
further comprising the act of following the at least one link. 
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method as recited in claim 1 wherein the audio representation includes 
only asjisaid links included in the electronic document. 
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Jfc. A method -as recited in claim 1 wherein the audio representation includes a 
;ount of said any links and aay. said text included in the electronic document. 

16. A method as recited in claim 1 wherein the electronic document is written in 
i markup language. 



U. A method as recited in claim Vi wherein the markup language includes 
elements that ^alxJ^capable of describing the visual appearance of the electronic 
document. 

/ P 

]p. A method as recited in claim L4 wherein the markup language is one of 
hypertext markup language and extensible markup language. 

16. A methoid as recited in claim 1 further comprising the act of providing the 
user with a visual representation of the electronic document simultaneously with the act of 
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In an information system that has access to electronic documents and is 
connected to a client by a telephony infrastructure, a method for providing the user with 
telephone access to an electronic document, the method comprising the acts of: 

parsing the Electronic document to identify any text and any links included in 
the electronic document; 

mapping gaid any text and said any links included in the electronic document 

more categories; and 
generating an audio^pepjresentation of at least a portion of the parsed and 
mapped electronic document, /the audio representation being communicated t 
cliej 

18. A method as recited in claim 17 further comprising the act of prompting the 
client to select one of the one or more categories. 

A mathod as recited in claim 1 8 further comprising the acts of: 

l instruction from the client, the instruction selecting one of the 
e or mori categories; and 

profnpting the client to select at least one link from the selected category. 

A jnethod as recited in claim 17 wherein the act of mapping produces exactly 
one category, further comprising the act of automatically selecting the one category. 
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21. A method i 
hierarchy of one or more c 
one or more levels. 



c/ted in claim 17 wherein the act of mapping produces a 
;s each having one or more links, the hierarchy including 



22. A method as recited in claim 21 further comprising the acts of: 
prompting the client to traverse the hierarchy of one or more categories and 

associated one or more links, automatically selecting any category and any 
associated link wheil exactly one category and exactly one associated link is present; 

receiving onjs or more instructions from the client to traverse the hierarchy, at 
least one instruction selecting an electronic document with text; and 

generating k audio iepresteritation of each part of the hierarchy that is visited, 
the audio representation of the electronic document with text including a count of 
any links and any/text present in the document. 

23. A method as recited in claim 17 further comprising the act of receiving an 
instruction from the clie it, wherein the instruction is spoken into the telephone. 

as recited in claim 23 wherein the instruction received from the 
;lient is at least one of an instruction to email, fax, or voice mail at least a portion of the 
electronic document to a third party selected from the client's contact list. 



25. A method as recited in claim 17 wherein the inforn 
electronic document ofer the Internet. 



ti system obtains the 



A method as recited in claim 17 wherein the el< 



a markup language and the 
describing the visual appearance 



document is written in 
-kup language includes elements that only are capable of 
f the electronic document. 



27. A method as ricited in claim 26 wherein the markup language is one of 
hypertext markup language anil extejlsfble markup language. 

A method as ijecite/l in claim 17 further comprising the act of providing the 
ith a visual representation of the electronic document simultaneously with the act of 
communicating the audio representation. 

29. A method as recited in claim 17 further comprising the acts of: 

receiving a request from the client to access the electronic document; and 
obtaining tha electronic document. 
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^3j>r In an information system that has access to electronic documents and is 
connected to a client by a telephony infrastructure, a method for providing the user with 
telephone access to an electronic document, the method comprising the acts of: 

receiving a reque it from the client to access the electronic document; 

obtaining the ele> :tronic document; 

parsing the elect onic document to identify any text and any links included in 



the electronic documentl and 
a 



generating 
electronic document, th 



31. A method as : 
any said text and a count of Si 



udio representation of at least a portion of the parsed 
audio representation being communicated to a client. 

cited ran clum 30 wherein the audio representation includes 
any links included in the electronic document. 



32. A method as recited in claim 30 further comprising the act of receiving a 
instruction from the client. 



33. A method as tecited in claim 32 wherein the instruction is spoken into the 



34. A method as recited in claim 33 wherein the instruction received from the 
client is at least one of an Instruction to email, fax, or voice mail at least a portion of the 
electronic document to a third party selected from the client's contact list. 
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35. A method as 
least one link and wherein 
further comprising the act of 



:cited in claim-33 wherein the audio representation includes at 
trje instruction from the client is to select the at least one link, 
bllowing the at least one link. 



36. A method as (recited in claim 30 wherein the audio representation includes 
6 only any said links includedjin the electronic document. 

37. A method ai recited in claim 36 wherein the audio representation includes 
one or more links, further comprising the act of prompting the client to select one of the one 
or more links. 



4f4l 



38. A method 
a markup language and tl 
describing the visual appearance of the electronic document. 



ited in claim 30 wherein the electronic document is written in 
arkup language includes elements that are only capable of 



l 30 wherein the markup language is one of 



hypertext markup languagb and extensible markup language. 



40. A methoc 
with a visual repre< 
communicating the audi ) 



as recited in claim 30 further comprising the act of providing the 
entation of the electronic document simultaneously with the act of 
representation. 
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In an information system that has access, through the Internet, to one or more 
documents, wherein the ofie or more electronic document^ are written in a markup 
language having elements that onb} are capable of describing the visual appearance of the 
electronic documents' and the information system is connected to a client by a 
telephony infrastructure, a method for providing the user with telephone access to the one 
more electronic documents, the method comprising the acts of: 

obtaining one o/'the one or more electronic documents; 
parsing the ohe electronic document to identify any text and any links 
included in the one electronic document; and 

generating ah audio representation of at least a portion of the parsed c 
electronic document, the audio representation being communicated to the cliei 

42. A method aa(repfte|d in claim 41 further comprising the act of mapping said 
any text and said any links/irfcluded in the electronic document to one or more categories. 
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receiving 
least one instruction/: 
generating 



43. A method as reiited in claim 41 wherein the act of mapping produces a 
hierarchy of one or more categories each having one or more links, the hierarchy including 
one or more levels, further comprising the acts of: 

prompting the fclient to traverse the hierarchy of one or more categories and 
associated one or more Ukks, automatically selecting any category and any 
associated link when/exacwohi category and exactly one associated link is present; 

more instructions from the client to traverse the hierarchy, at 
n selecting an electronic document with text; and 
a audio representation of each part of the hierarchy that is visited, 
the audio represent ition of the electronic document with text including a count of 
any links and any tejxt present in the document. 

44. A method as recitid in claim 41 further comprising the act of receiving an 
instruction from the client, wherain the instruction is spoken into the telephone. 

45. A method as ricited in claim 44 wherein the instruction received from the 

> / 

ient is at least one of an ir/struction to email, fax, or voice mail at least a portion of the 

electronic document to a thirti party selected from the client's contact list. 

46. A method is recited in claim 44 wherein the audio representation includes at 
least one link and wherain the instruction from the client is to select the at least one link, 
further comprising the afct of following the at least one link. 



4^ Armelhod as recited in claim 41 further comprising The act of providing the 
r with a visual representation of the electronic document simultaneously with the act of 
communicating the audio representation. 
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In a system that includes an information service and an input/output device 
for interacting with the information service, a method providing an audio interface to 
navigating between a plurality of electronic documents, the method comprising the acts of: 

obtaining a first electronic document; 

parsing the first/ electronic document to identify any text and any links 
included in the first electronic document; 

generating atf first audio representation of at least a portion of the parsed first 
electronic document, the first audio representation being communicated to a client; 

obtaining a secc nd electronic 

parsing the seond electvpmcldocument to identify any text and any links 
included in the second electronic document; and 

making any lirks identified in the first electronic document available for 



selection in the second 



electronic document. 



49. A method as r< 
mapping said 



50. A method a 
instruction from a client, w 



;ited in claim 48 further comprising the acts of: 

ny text and said any links included in the electronic document 



generating an a idio representation of the one o 



ted in claim 48 further comprising the act of receiving a 
l the instruction is spoken by the client. 
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A system p/oviding access, through an audio interface, to electronic 
documents, wherein the Electronic documents are written in a markup language having 
elements that only ari/ capable of describing the visual appearance of the electronic 
documents, the systemr comprising: 

processor means for obtaining an electronic document; 

processor means for parsing the electronic document to identify any text and 
any links' included in the electronic document; and 

processor means for generating an audio representation of at least a portion of 
the /parsed electronic document, the audio representation being communicated to a 



system as recited in claim 51 further comprising processor means for 
mapping sajd an^xt^d said any links included in the electronic document to one or more 
categories. 

A system as r/cited in claim 51 further comprising processor means for 
receiving a spoken instruction from the client. 

A system/as recited in claim 53 wherein the instruction received from the 
client is at least one of/an instruction to email, fax, or voice mail at least a portion of the 
electronic document td a third party selected from the client's contact list. 
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55. A system as recited/in claim 53 wherein the audio representation includes a 
least one link and wherein the ir/struction from the client is to select the at least one link 
further comprising processor means for following the at least one link. 
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A computer prjbgram product for providing access, through an audio 

interface, to electronic documents, wherein the electronic documents are written in a markup 

language having elements thai only are capable of describing the visual appearance of the 

electronic documents, the computer program product comprising: 

a computer-reidable medium carrying computer-executable instructions for 

implementing the method, the computer-executable instructions comprising: 

program code means for obtaining an electronic document; 

progr im code means for parsing the electronic document to identify 

any text and my links included in the electronic document; and 

progiam code means for generating an audio representation of at least 

a portion of the parsed electronic document, the audio representation being 

communicated to a client. 
/ 

57. A computer+eadable medium as recited in claim 56 wherein the computer- 
executable instructions fufthe^coJnprises program code means for mapping said any text 
and said any links included id the Electronic document to one or more categories. 

A computer-readable medivfm as recited in claim 56 wherein the computer- 
executable instructions further comprises program code means for receiving a spoken 
instruction from the client. 

59. A computer-readabAe medium as recited in claim 58 wherein the instruction 
;ceived from the client is at leist one of an instruction to email, fax, or voice mail at least £> 
portion of the electronic document to a third party selected from the client's contact list. 
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60. A computer-readable medium as recited in claim 58 wherein the audio 
representation includes at lea/t one link and wherein the instruction from the client is to 
select the at least one link, wherein the computer-executable instructions further comprises 
program code means for following the at least one link. 
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ABSTRACT OF THE INVENTION 

Methods, systems, and computer program products for providing an audio interface 
to electronic documents. An audio interface receives a request for electronic content from a 
client and retrieves the requested document. Next, the document is parsed to identify 
various elements, such as title, text, and links included in the document. Many of those 
providing electronic content include hierarchical indices to aid clients in finding documents 
covering a particular topic. The audio interface includes a facility to preserve, enhance, or 
create this hierarchical organization by mapping a document's text and links into a 
corresponding audio hierarchy. The client is then presented with various options from 
which to make a selection. Outside of this hierarchy, the client receives the document title, 
the number of links contained within the document, an audio representation of the 
document's text, and may optionally chose to hear only the links included in the document. 
The audio interface also includes a number of global commands to facilitate navigation. 
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DECLARATION CLAIMING SMALL ENTITY STATUS 
FOR A SMALL BUSINESS CONCERN 

I, David L. Morton, hereby declare: that I am President of talk2.com, a corporation of the 

State of Nevada and having a principal place of business at 746 East Winchester Avenue, #210, 

Salt Lake City, UT 84107; I am empowered to act on behalf of talk2.com; and that talk2.com 

qualifies as a small business concern as defined in 13 C.F.R. § 121.3-18 and 37 C.F.R. § 1.9(d), 

for purposes of paying reduced fees to the Patent and Trademark Office under Section 41(a) and 

(b) of Title 35, United States Code, in that the number of employees of talk2.com, including 

those of its affiliates, does not exceed 500 persons. I understand that, for purposes of this 

declaration, (1) the number of employees is the average over the previous fiscal year of the 

number of persons employed on a full-time, part-time, or temporary basis during each of the pay 

periods of the fiscal year, and that (2) two business concerns are considered to be affiliates of 

each other when one business concern either directly or indirectly controls or has the power to 

control the other, or when a third party or parties control or have the power to control both 

business concerns. 

I further declare that all rights, title, and interest relating to the invention entitled "VOICE 
INTERFACE FOR ELECTRONIC DEVICE," invented by Darren L. Wesemann, Dong-Kyun 
Nam, and Richard T. Newton, as described in the patent application filed concurrently herewith 
have been conveyed to and currently remain with tallc2.com. 

I acknowledge the duty to file, in the above-mentioned application or any patent issued in 
respect thereof, notification of any change in status resulting in loss of entitlement to small entity 
status prior to paying, or at the time of paying, the earliest of (1) the issue fee or (2) any 



maintenance fee due after the date on which status as a small entity is no longer appropriate. I 
further declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful, false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States 
Code, and that such willful, false statements may jeopardize the validity of the application, any 
patent issuing thereon, or any patent to which this declaration is directed. 
Signed at Salt Lake City, Utah, this /j? day of December, 1999. 



talk2.com 




David L. Morton, 
President 
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Express Mail Label No. EL 446 775 357 US 



PATENT APPLICATION 
Docket No: 14999.3 



DECLARATION, POWER OF ATTORNEY, AND PE TITION 



We, 



(1) 



Address: 



Darren L. Wesemann 



Citizenship: 



229 North Fairway Drive 
North Salt Lake, UT 84054 
United States of America 



(2) 



Name: 
Address: 



Dong-Kyun Nam 
847 East 7440 South #20 
Midvale, UT 84047 
Korea 



Citizenship: 



Address: 



Richard T. Newton 
372 East 1310 South 
Kaysville, UT 84037 



Citizenship: 



United States of America 



declare: that our citizenship, residence address, and post office address are as set forth above; 
that we verily believe we are the original, first, and joint inventors of the subject matter of the 
invention or discovery entitled "VOICE INTERFACE FOR ELECTRONIC DOCUMENTS" for 
which a patent is sought and which is described and claimed in the specification attached hereto; 
that we have reviewed and understand the contents of the above-identified specification, 
including the claims, as amended by any amendment specifically referred to herein; and that we 
acknowledge the duty to disclose information which is material to the patentability of this 
application in accordance with Section 1.56(a) of Title 37 of the Code of Federal Regulations. 

We declare further that all statements made herein of our own knowledge are true and 
that all statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful, false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States 



Code, and that such willful, false statements may jeopardize the validity of the application or any 

patent issuing thereon. 

We hereby appoint as our attorneys and/or patent agents: RICK D. NYDEGGER, 

Registration No. 28,651; DAVID O. SEELEY, Registration No. 30,148; JONATHAN W. 

RICHARDS, Registration No. 29,843; JOHN C. STRTNGHAM, Registration No. 40,831; 

BRADLEY K. DeSANDRO, Registration No. 34,521; JOHN M. GUYNN, Registration 

No. 36,153; CHARLES L. ROBERTS, Registration No. 32,434; GREGORY M. TAYLOR, 

Registration No. 34,263; DANA L. TANGREN, Registration No. 37,246; KEVIN B. 

LAURENCE, Registration No. 38,219; ERIC L. MASCHOFF, Registration No. 36,596; C. J. 

VEVERKA, Registration No. 40,858; ROBYN L. PHILLIPS, Registration No. 39,330; 

RICHARD C. GILMORE, Registration No. 37,335; DAVID B. DELLENBACH, Registration 

No. 39,166; KEVIN K. JOHANSON, Registration No. 38,506; DAVID L. GRIFFIN, 

Registration No. 44,136; R. BURNS ISRAELSEN, Registration No. 42,685; DAVID R. TODD, 

Registration No. 41,348; JESUS JUAN6S i TIMONEDA, Registration No. 43,332; STEPHEN 

D. PRODNUK, Registration No. 43,020; R. PARRISH FREEMAN, JR., Registration No. 

42,556; ADRIAN J. LEE, Registration No. 42,785; and KYLE H. FLINDT, Registration No. 

42,539, with full power of substitution and revocation, to prosecute this application and to 

transact all business in the Patent and Trademark Office connected therewith. AH 

correspondence and telephonic communications should be directed to: 

R. Burns Israelsen 
WORKMAN, NYDEGGER & SEELEY 
1000 Eagle Gate Tower 
60 East South Temple 
Salt Lake City, Utah 84111 
Telephone: (801) 533-9800 
Facsimile: (801) 328-1707 

2 



Wherefore, we pray that Letters Patent be granted to us for the invention or discovery 
described and claimed in the foregoing specification and claims, declaration, power of attorney, 
and this petition. 

Dated this AT"day of ^2^^, 19' 



Inventor: f fas^tCfaut,^ 

/ / Darxen L. Wesemann 
^ 229 North Fairway Drive 
North Salt Lake, UT 84054 



Dated this 13 day of Dft/Cfyfkl^ , 1999. 



Inventor: 




Dong-Kyun Nam 

847 East 7440 South #20 

Midvale, UT 84047 



Dated this ]5_ day of D^ff/A/fl999. 



Inventor: 




Richard T. Newton 

229 North Tail w a y Dri ve ff7£r3flST |3|D 
NeA&rit^ akc, UT 840 54 ft*yQ/i||e f [Af 
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Guide to Filing a Utility Patent Application- 



INTRODUCTION 



-1210 



The U.S. Patent and Trademark Office (PTO) is the government agency responsible for examining 
patent applications and issuing patents. A patent is a type of property right. It gives the patent holder 
the right, for a limited time, to exclude others from making, using, or selling the subject matter that is 
within the scope of protection granted by the patent The PTO determines wether a patent should be 
granted in particular case. However, it is up to the patent holder to enforce his or her own rights if 
the PTO does grant a patent, 

The purpose of this guide is to provide you with basic information about filing a utility patent 
application. A patent application is a complex legal document, best prepared by one trained to 
prepare such documents. Thus, after reviewing this guide, you may wish to consult with a patent 
attorney or agent Additional information is available: 

• by calling the PTO's General Information Services at 800-PTO-9199 or 703-308-4357, 

• from the PTO's Web site at www.uspto.gou nd — 1220 

• at your nearest Patent and Trademark Depository Library (PTOL). You will find information 

• regarding the nearest PTOL at the end of this guide. 1230 

There are various types of patents - utility, design, and plant There are also two types of utility 
patent application - provisional and nonprovisional. Each year the PTO receives approximately 
200,000 patent applications. Most of these are for nonprovisional utility patents. 

This guide contains information to assist you in filing your nonprovisional utility patent application. 
It discusses the required parts of the utility patent application and includes samples of some of the 
forms you may use. This information is generally derived from the Patent Act, found at Title 35 of 
the United States Code (U.S.C.), and Title 37 of the Code of Federal Regulations (CFR). These 
materials are available atPTOLs and at most law libraries. 
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Express Mail Label No. EL 550 

FormPTO-1 4 19 

Applicant: Darren L. Wesemaif 

Serial No, 09/464,989 U '''^ J: Att* Dock* No, 149993 

FilmgDate: December 16, 1999 % / Group- 2748 

For: VOICE INTERFACE I%l»E%TRONIC DOCUMENTS 

INFORMATION DISCLOSURE CITATIONS MADE BY APPLICANT 

U.S. Patent Documents ° 




Examiner Patent Issue 



3E TO 



f ... . j, , Sub Filing <— 

IffltiaP Number Date Name Class Class Date g 

!2^f_Al 5,799,063 08/25/98 Krane 379 J^ggM 08/1 5/g 

2i£_A2 5,884,262 . ; 03/16/99 Wise et al. 704 270.1 lo3/28/I, 

References Cited by Applicants 



While the filing of Information Disclosure Statements is voluntary, the procedure is 

C°fT 88 V7 f ■ \ o m T °l SeCti °!J ° f ^ ManUal ° f Pat6nt * X ™ * -d 37 

i^n u Z considered a proper Information Disclosure Statement Form PTO- 

1449 shall be accompanied by a copy of each listed patent or publication or other item of 
mformat.on and a translation of the pertinent portions of foreign documents (if an existing 
ranslation is readily avadable to the applicant), an explanation of relevance of each reference not 
the English language, and should be submitted in a timely manner as set out in MPEP Sec. 609. 

MPFP S a ino rS ™\ COnsider a" cations submitted in conformance with 37 C.F.R. § 1.98 and 
MPEP Sec. 609 and place their initials adjacent the citations in the spaces provided on this form 
Examiners wiU also initial citations not in conformance with the guidelines which may have bTn 

cStT, '• fin 6 ^ ^ uT dered by EXaminer f ° r reason whether °r ** the 

citation is m full conformance with the guidelines. A line will be drawn through a citation if it is 
not in conformance with the guidelines AND has not been considered. A copy of the submitted 

Th?on^r^ / ^r 11 "' WiU bC retUmed t0 the a PP licant with the next communication 
The ongmal of the form will be entered into the application file. 

Each citation initialed by the Examiner will be printed on the issued patent in the same 
m anner as references cited by the Examiner onFormPTO-892. 

The reference designations "Al," "A2," etc. (referring to Applicant's reference 1 
Applicant's reference 2 eta) will be used by the Examiner in the same manne a sT^nJs 
reference designations "A," "B," "C," etc. on Office Action Form PTO-1 142. ^miners 
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mark Office. U.S. DEPARTMENT OF COMMERCE 



Express Mail Label No. EL 550 336 641 US ^ 

m THE UNITED STATES PATENT AND TRADEMARK OFFICE 




PATENT APPLICATION 
Docket No: 14999. 



In re application of 



Serial No.: 
Filed: 




Darren L. Wesemann, et al. 

09/464,989 

December 16, 1999 

For: VOICE INTERFACE FOR 

ELECTRONIC DOCUMENTS 

INFORMATION DISCLOSUR E STATEMENT t < 0 O 

UNDER37C.F.R. S 1.97 :.' C: ' J 

Assistant Commissioner for Patents g O 

Washington, D C. 20231 g 

Sir: * ' 

Please find, pursuant to 37 C.F.R. § 1.98(a)(1), the enclosed Form PTO-1449 which 

contains a list of all patents, publications, or other items that have come to the attention of one or 

more of the individuals designated in 37 C.F.R. § 1.56(c). While no representation is made that 

any of these references may be "prior art" within the meaning of that term under 35 U.S.C. §§ 

102 or 103, the enclosed list of references is disclosed so as to fully comply with the duty of 

disclosure set forth in 37 C.F.R. § 1.56. 

Moreover, while no representation is made that a specific search of office files or patent 

office records has been conducted or that no better art exists, the undersigned attorney of record 

believes that the enclosed art is the closest to the claimed invention (taken in its entirety) of 

which the undersigned is presently aware, and no art which is closer to the claimed invention 

(taken in its entirety) has been knowingly withheld. 



1 



In accordance with 37 C.F.R. §§ 1.97 and 1 
relevant portion thereof is also enclosed. 
Dated this 2.6 jay of MAfofrl 



I, a copy of each of the listed references or 



, 2000. 

Resgectfiilly submitted, 

_ <:d.nydegger/^ 

Attorney for Applicant 
Registration No. 28,651 

WORKMAN, NYDEGGER & SEELEY 
1000 Eagle Gate Tower 
60 East South Temple 
Salt Lake City, Utah 84111 
Telephone: (801) 533-9800 
Facsimile: (801) 328-1707 



2 




VOICE INTERFACE FOR ELECTRONIC DOCUMENTS 



hereby certify that the Transmittal Letter (in triplicate); Information Disclosure 
Statement under 37 C.F.R. § 1.97 (2pgs); Form PTO-1449 (1pg); Legible copies of two (2) 
references; and postcard are being deposited with the United States Postal Service 
"Express Mail Post Office to Addressee" service under 37 C.F.R. § 1.10 in an envelope 
addressed to: The Assistant Commissioner of Patents, Washington, D.C. 20231 on 
MAfiCH 2000. 

DACIA SMITH 



EL 550 336 641 US 



. i — EXPRESS MAIL LABEL NO. : 550 336 641 US ^ MENT APPLICATION 



WORKMAN 

NYDEOBER „„„,„_. „„ 
<s?SEELEY r™:™" 05 oaS s lt r r d ee r d oy y^oTp^\ TRADE SECRETS 




'ATENT 
TRADEMARK 
COPYRIGHT 



COMPETITION 
J\ LICENSING 
^COMPLEX LITIGATION 



IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
TRANSMITTAL FOR INFORMATION DISCLOSURE STATEMENT Z 



Assistant Commissioner for Patents 
Washington, DC 20231 



Transmitted herewith for filing and pursuant to 37 C.F.R. § 1.97 - is an Information 
Disclosure Statement. I 



Enclosed also are the following designated documents, as required under 37 C.F.R. §§ 1.97 
and 1.98: 



X Form PTO-1449 list of two (2) references submitted for consideration. 

X Legible copies of the listed references or their relevant portions. 

All English translations of each nonenglish reference, if any, within the possession, 

custody, control or availability of anyone designated in 37 C.F.R. § 1.56(c) (see 37 
C.F.R. § 1.98(c)). 

The following are included within the Information Disclosure Statement if applicable and as 
required under 37 C.F.R. § 1.98: 

Concise explanation of relevance of each reference not in English and 

unaccompanied by an English translation. 

Statement that certain listed references not enclosed are substantially cumulative of 

an enclosed reference. 



Statement that certain listed references not enclosed were previously cited by or 
submitted to the Office in the identified prior application which is relied upon for an 
earlier filing date under 35 U.S.C. § 120. 



In order to secure coria. aeration of the items designated abo.j, one or more of the 
following, if required, is also enclosed: 



Promptness Certification. 



Check No. in the amount of $240.00 constituting submission fee under 

37 C.F.R. 1.17(p); 

Petition for Consideration and Check No. in the amount of 



X In the event that 37 C.F.R. § 1.97(c) applies and the Examiner is not satisfied that 
the Promptness Certification meets the requirements of 37 C.F.R. § 1.97(e), or in 
any other event remediable by a fee, please credit any over payment or charge any 
additional fees to Deposit Account No. 23-3 178 of the undersigned. 



Dated this 2ft day of Mftf^ 2000. 




CKD.NYDEGGER' 
""Attorney for Applic; 
Registration No. 28,651 

WORKMAN, NYDEGGER & SEELEY 
1000'Eagle Gate Tower 
60 East South Temple 
Salt Lake City, Utah 84111 
Telephone: (801) 533-9800 
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FormPTO-1449 , 

' Applicant: Darren L. Wesemanh, eW. 2 B 

SerialNo.: 09/464,989 

FOingDate: ^^Z^M^KOmC^OCUMEmS 



Att'y Docket No.: 14999.3 
Group: 2748 



Examiner Patent 
Initial* Number 



nA^_ Al 5,799,063 
2$_ A2 5,884,262 



Issue 
Date 



08/25/98 
03/16/99 



Name 
Krane 
Wise et al. 



Sub 
Class Class 



Filing f- 
Date 1 



08/15/96 
: 03/28/96 



References Cited by Applicants 



translation is readily available to the applicant), an explanation ot reievdnc 

SSeSi language, and should be submitted in a timely manner as set out in MPEP Sec. 609. 

citation is in full conformance with the guidelines. A lm w^ be d awn g ^ 

The original of the form will be entered into the application file. 

Each citation initialed by the Examiner win be printed on the issued patent in the same 
manner as references cited by the Examiner on Form PTO-892. 

reference designations "A," "B," "C," etc. on Office Action Form PTO 1 142. 



Date Considered: 



conformance with MPEP 609; draw 



* EXAMINER: ^^^^^^^^^ 
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UNITED STATES „c'PARTMENT OF /COMMERCE 



Patent and Trademark Office 

ASSISTANT SECRETARY AND COMMISSJL.^-. . 

^TRADEMARKS (J^k)? CJ/jOU 



CHANGE OF ADDRESS/POWER OF ATTORNEY 

• *\ 

FILE LOCATION 27C1 SERIAL NUMBER 09464989 PATENT NUMBER %_ ^a> ^ 

THE CORRESPONDENCE ADDRESS HAS BEEN. CHANGED TO CUSTOMER # 22913 "\ % ^ 

THE PRACTITIONERS OF RECORD HAVE BEEN CHANGED TO CUSTOMER # 22913 ^q, 

THE FEE ADDRESS HAS BEEN CHANGED TO CUSTOMER # 22913 

ON 08/31/00 THE ADDRESS OF RECORD FOR CUSTOMER NUMBER 22913 IS: 

WORKMAN NYDEGGER & SEELEY 
1000 EAGLE GATE TOWER 
60 EAST SOUTH TEMPLE 
SALT LAKE CITY UT 84111 



AND THE PRACTITIONERS OF RECORD FOR CUSTOMER NUMBER 22913 ARE: 

28651 29843 30148 32434 34263 34521 36153 36596 37246 37335 
38219 38506 39166 39330 40831 40858 41348 42539 42556 42685 
42785 43332 44136 44515 45454 45576 45666 46081 



PTO INSTRUCTIONS: PLEASE TAKE THE FOLLOWING ACTION WHEN THE 
CORRESPONDENCE ADDRESS HAS BEEN CHANGED TO CUSTOMER NUMBER: 
RECORD, ON THE NEXT AVAILABLE CONTENTS LINE OF THE FILE JACKET , 
'ADDRESS CHANGE TO CUSTOMER NUMBER' . LINE THROUGH THE OLD 
ADDRESS ON THE FILE JACKET LABEL AND ENTER ONLY THE ' CUSTOMER 
NUMBER' AS THE NEW ADDRESS. FILE THIS LETTER IN THE FILE JACKET . 
WHEN ABOVE CHANGES ARE ONLY TO FEE ADDRESS AND/OR PRACTITIONERS 
OF RECORD, FILE LETTER IN THE FILE JACKET. 
THIS FILE IS ASSIGNED TO GAU 2742. 



PTO-FMD 
TALBOT- 1/97 



/ V \ UNITED STATES DEPARTMENT OF COMMEh. 

* "SU 1 Patent and Trademark Office 

\WSZ$ J Address- COMMISSIONER OF PATENTS AND TRADEMARKS 
°<^Z<i>* Washington, D.C. 20231 




P ,ease find below and/or attached an Office communication concerning this appiication or 
proceeding. 



Commissioner ot Patents and Trademarks 



Office Action Summary 



Applicatioi 

09/464, 98S 



Applicant(s)- 

WESEMANN E 



irrespondence address 



ion appears on the c( 



lication(s) filed on 16 December 1999 . 
2a)D This action is FINAL 2b)rx] This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

isposition of Claims 

4) [X] Claim(s) 1-60 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) U50 is/are rejected. 

7) [x] Claim(s) 11,12 and 48 is/are objected to. 

8) 0 Claims are subject to restriction and/or election requirement. 



Application Papers 

9)D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are objected to by the Examiner. 

11) D The proposed drawing correction filed on is: a)D approved b)D disapproved, 

12) Q The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. § 119 

13) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)QAII b)Q Some*c)D None of: 

1 □ Certified copies of the priority documents have been received. 

2.Q Certified copies of the priority documents have been received in Application No. . 

3-D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgement is made of a claim for domestic priority under 35 U.S.C. § 1 19(e). 



Attachment(s) 

1 5) [X] Notice of References Cited (PTO-892) 1 8) □ Interview Summary (PTO-41 3) Paper No(s). . 

16) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 19) □ Notice of Informal Patent Application (PTO-152) 

1 7) E3 Information Disclosure Statement(s) (PTO-1 449) Paper No(s) . 20) □ Other: 

PTO-326 (Rev, 01-01) Office Action Summary Part of Paper No. 4 



Application/Control Number: 09/464,989 
Art Unit: 2645 



Page 2 



DETAILED ACTION 
Claim Objections 

1 . Claims 11,12 and 48 are objected to because of the following informalities: In the 
second line of claim 1 1 , and in the second line of claim 1 2, the words "any said" should be 
reversed to read "said any". Also, in the second line of the claim 48, the word "to" following 
"interface" should be changed to "for" or alternatively, the word "navigating" should be changed 
to "navigate". Also in claim 48, in the seventh line of the claim, the word "an" following 
"generating" should be changed to "a". Appropriate correction is required. 

Claim Rejections - 35 USC §102 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in the United 
States before the invention thereof by the applicant for patent, or on an international application by another who 
has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this title before the invention 
thereof by the applicant for patent. 

3. Claims 1-7, 9-23, 25-33, 35-44, 46-53, 55-58 and 60 are rejected under 35 U.S.C. 102(e) 
as being anticipated by Wise et al. (US 5884262, March 1999). 

4. Regarding claims 1 drawn to a system that includes an information service and an 
input/output device for interacting with the information service, applicant claims a method of 
providing an audio interface for accessing electronic documents, the method comprising the acts 
of: obtaining an electronic document; parsing the electronic document to identify any text and 



Application/Control Number: 09/464,989 Page 3 

Art Unit: 2645 

any links included in the electronic document; and generating an audio representation of at least 
a portion of the parsed electronic document. Wise et al. teaches providing an audio interface for 
accessing electronic documents, the method comprising the acts of: obtaining an electronic 
document; parsing the electronic document to identify any text and any links included in the 
electronic document; and generating an audio representation of at least a portion of the parsed 
electronic document (See abstract and column 2 lines 5-33). 

5 . Regarding claims 2 and 48 drawn to the invention of claim 1 , applicant further claims 
obtaining at least two electronic documents, one of the at least two electronic documents being 
designated as a prior electronic document and one of the at least two electronic documents being 
designated as a subsequent electronic document (or as stated in claim 48, obtaining a first and 
second electronic document); parsing the at least two electronic documents to identify any text 
and links included in the at least two electronic documents (or as stated in claim 48, parsing the 
first and then the second electronic document); and making any links identified in the prior 
electronic document available in the subsequent electronic document (where the audio 
representation of claim one relates to the first document). Wise et al. teaches obtaining at least 
two electronic documents, one of the at least two electronic documents being designated as a 
prior electronic document and one of the at least two electronic documents being designated as a 
subsequent electronic document (or as stated in claim 48, obtaining a first and second electronic 
document) (See column 2 lines 59-62); parsing the at least two electronic documents to identify 
any text and links included in the at least two electronic documents (or as stated in claim 48, 
parsing the first and then the second electronic document) (See column 2 lines 64-67 and column 
5 lines 66-67 and column 6 lines 1-14); and making any links identified in the prior electronic 
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document available in the subsequent electronic document (where the audio representation of 
claim one relates to the first document) (See column 6 lines 1-14). 

6. Regarding claims 3, 7, 12-13, and 16 drawn to the invention of claim 1, applicant claims 
further the act of mapping said any text and said any links included in the electronic document to 
one or more categories; the act of receiving an instruction from the client; and the act of 
providing the user with a visual representation of the electronic document simultaneously with 
the act of communicating the audio representation; and wherein the audio representation includes 
a count of said any links and said any text included in the electronic document; and wherein the 
electronic document is written in markup language. Wise et al. teaches the act of mapping said 
any text and said any links included in the electronic document to one or more categories (See 
column 2 lines 18-29 and column 5 line 67 and column 6 lines 1-4 and column 7 lines 13-27); 
the act of receiving an instruction from the client (See column 2 lines 42-50); and the act of 
providing the user with a visual representation of the electronic document simultaneously with 
the act of communicating the audio representation (See column 8 lines 50-54); and wherein the 
audio representation includes a count of said any links and said any text included in the 
electronic document (See column 6 lines 52-64); and wherein the electronic document is written 
in markup language (See column 2 lines 9-1 1 - where HTML is an exemplary markup language). 

7. Regarding claims 4 and 6 drawn to the invention of claims 1 and 3, applicant claims 
further the acts of generating an audio representation of the one or more categories; and wherein 
the act of mapping includes creating a hierarchy of said any text and said any links. Wise et al. 
teaches the acts of generating an audio representation of the one or more categories (See column 
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2 lines 1 8-29); and wherein the act of mapping includes creating a hierarchy of said any text and 
said any links (See column 7 lines 13-27). 

8. Regarding claim 5 drawn to the invention of claims 1, 3 and 4, applicant further claims 
the act of prompting the client to select one of the one or more categories. Wise et al. teaches 
prompting the client to select one of the one or more categories (See column 2 lines 43-48 and 
column 6 lines 58-67). 

9. Regarding claims 9-1 1 drawn to the invention of claims 1 and 7, applicant claims further 
the instruction received from the client is spoken into the input/output device; and wherein the 
audio representation includes at least one link and wherein the instruction from the client is to 
select the at least one link, further comprising the act of following the at least one link; and 
wherein the audio representation includes only any said links included in the electronic 
document. Wise et al. the instruction received from the client is spoken into the input/output 
device (See column 2 lines 42-50); and wherein the audio representation includes at least one 
link and wherein the instruction from the client is to select the at least one link, further 
comprising the act of following the at least one link (See column 7 lines 7-33); and wherein the 
audio representation includes only any said links included in the electronic document (See 
column 7 lines 2-6). 

10. Regarding claims 14 and 15 drawn to the invention of claims 1 and 13, applicant further 
claims the markup language includes elements that only are capable of describing the visual 
appearance of the electronic document; and wherein the markup language is one of hypertext 
markup language and extensible markup language. Wise et al. teaches the markup language 
includes elements that only are capable of describing the visual appearance of the electronic 
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document (See column 3 lines 19-24); and wherein the markup language is one of hypertext 
markup language and extensible markup language (See column 2 lines 7-1 1 and column 7 lines 
28-33). 

1 1 . Regarding claim 20, drawn the invention of claim 17, applicant further claims wherein 
the act of mapping produces exactly one category, and further comprising the act of 
automatically selecting the one category. Wise et al. teaches the act of mapping producing 
exactly one category, and further comprising the act of automatically selecting the one category 
(See column 7 lines 7-11). 

12. Regarding claims 17-19 and 21-23, 25-33, 35, 37-44, 46-53 and 55-58 and 60, all 
elements of the claimed limitations are disclosed in the above rejections and therefore these 
claims (considered separately or in any combination) are also rejected as anticipated by Wise et 
al. 

Claim Rejections - 35 USC §103 

13. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject'matter pertains. Patentability shall not be negatived by the 

14. Claims 8, 1 1, 24, 34, 36, 45, 54, and 59 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Wise et al. in view of Rhie et al. (US 5953392, September 1999). 

15. Regarding claims 8, 24, 34, 45, 54, and 59 drawn to the inventions of claims 1, 7, 17, 23, 
41, 44, 51, 53 and 56, applicant claims wherein the instruction received from the client is at least 
one of an instruction to email, fax, or voice mail at least a portion of the electronic document to a 
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third party selected from the client's contact list. Wise et al. teaches all the limitations of claims 
1, 7, 17, 23, 41, 44, 51, 53, and 56 as described in paragraphs 4, 6 and 12 above. Wise et al. 
further teaches the instruction received from the client is to fax at least a portion of said 
electronic document to the user's facsimile machine or computer. Wise does not teach the client 
sending at least a portion of the electronic document to a third party selected from the client's 
contact list. Rhie et al. teaches a method and apparatus for telephonically accessing and 
navigating the internet with a document delivery subsystem which provides a method for 
delivering an electronic document via one of the well known and available methods such as 
voice, fax-on-demand, electronic mail or regular mail (See abstract - last 5 lines, and column 6 
lines 23-33). One of ordinary skill in the art at the time the invention was made would have been 
motivated to combine the teaching of Rhie, providing multiple flexible methods of delivering 
information accessed by telephony device, with the teaching of Wise for the desirable purpose of 
providing a method of accessing and retrieving information from interconnected networks for 
those who have limited hardware and/or software or who have no means other than telephone to 
do so (See Rhie, column 1 lines 34-42). 

16. Regarding claims 1 1 and 36 drawn to the invention of claims 1 and 30, applicant further 
claims the audio representation includes only said any links included in the electronic document. 
Wise et al. teaches all the limitations of claims 1 and 30 as described in paragraphs 4 and 12 
above. Wise does not explicitly teach the audio representation including only links of the 
electronic document. Wise does teach the user may select the amount of descriptive information 
recited (See column 7 lines 2-6). Rhie et al. teaches signaling the user in an audio manner the 
hyperlink selections in a web page (See column 2 lines 4-6). One of ordinary skill in the art at 
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the time the invention was made would have been motivated to combine the teachings of Rhie 
with the teachings of Wise (having the user select to hear only the links) for the desirable 
purpose of providing a method of accessing the hyperlinks of a web page in an inexpensive and 
efficient manner (See Rhie column 1 lines 47-62). 

Conclusion 

1 7. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. Kawamata (US 6122290, September 2000), Perrone (US 6157705, December 2000) 
and Uppaluru (US 5915001, June 1999) are cited as further works in the related filed of voice 
browsers and universal access to the internet and other electronic resources 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Nora J Putt whose telephone number is 703-308-4736. The 
examiner can normally be reached on M-F 0700-1530. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Fan Tsang can be reached on 703-305-4895. The fax phone numbers for the 
organization where this application or proceeding is assigned are 703-308-5403 for regular 
communications and 703-308-5403 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 703-305-4700. 



NJP 

February 16, 2001 



FAN TSANG 
SUPERVISORY PATENT EXAMINER 
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Workman, Nydegger &l Seeley 

1000 Eagle Gate Tower 
60 East South Temple 
Salt Lake City, Utah 84111 
Phone: (801) 533-9800 
Fax: (801) 328-1707 

FAX TRANSMISSION COVER SHEET 



Date: April 16,2001 

To . NORA J, PUTT 

Fax: 703-308-5403 

Phone: 703-308-4736 

Rc: VOICE INTERFACE FOR ELECTRONIC DOCUMENTS 

Sender: r. BURNS ISRAELSEN 



YOU SHOULD RECEIVE _5_ PAGE(S), INCLUDING THIS COVER SHEET. IP YOU DO 

NOT RECEIVE ALL THE PAGES, PLEASE CALL 801-533-9800 



P RIVILEGE D AINU CO NFIDENT IAL 



TUB INFORMATION CONTAINED IN THIS 
INFORMATION INTENDED ONLY FOR THE USE OF 1 
MESSAGE IS NO T THE INTENDED RECIPIENT, OU 
INTENDED RECIPIENT, YOU ARE HEREBY NOTIF1E 
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Proposed Interview Agcndn - Do Not Enter In Record 

Examiner Putt, N. 
Patent Application Serial No. 09/464,989 
VOICE INTERFACE FOR ELECTRONIC DOCUMENTS 
Filed: December 16, 1999 
Attorney Docket No. 14999.3 
Attorney: R. Burns Jsraelsen 
Today's Date: April 16, 2001 



In preparation for a requested Examiner Interview, and as requested by the Exomii 
Applicants propose the following agenda: 



I. Attorney to present brief summary of invention. 

II. Attorney to discuss rejected claims and cited art, including the following 
independent claims: 

A. Claims 1, 17, 30, 41, 48, 51 and 56, rejected under 35 U.S.C, § 102(c) as 
being anticipated by U.S. Patent No'. 5,684, 262 to Wise; 

III. Attorney to present proposed amended claims and discuss how proposed amended 
claims overcome rejections and distinguish from art of record, A copy of the a 
proposed amended claim 1 accompanies this agenda. 



IV. Attorney to discuss any formal matters. 



Any questions regarding this proposed agenda can be directed to R. Burns Israelson 
(801) 533-9800 (phone) or (801) 328-1707 (fax). 
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Proposed Amended Claims - Do Not Enter In Record 
Patent Application Serial No. 09/464,989 
VOICE, INTERFACE FOR ELECTRONIC DOCUMENTS 
Filed: December 16, 1999 
Attorney Docket No. 14999.3 
Examiner Putt, N. 
April 16, 2001 

1. (Amended) In a system that includes an information service and a telephone 
[an input/output device] for interacting with the information service, a method of en abling a u ser 
nfjhgjglephono to access and navigate [providing an audio interface for accessing] electronic 
documents byj^rcsenting to the user an audio re presentatiorLo|^hjgrarehy„of links of Jhe 
document so as to p.nhaner. the ability of t he user to naviaatc the electronic doc uments, the 
method comprising the acts of: 

obtaining an electronic document; 

parsing the electronic document to identify any tort and any links included in the 
content of the electronic document; [and] 

( mapping content of the parsed electronic document 1 by p erforming t he acts of: 
j, (j - de termining /whethcrUhe text and links included in the con te nt of the 

/, , do cument rcpresen^categoriesr-first-leyel links and second-level li nks; and 

> u f >J y J to the extent that the text and links represent ca t egories, fi rst-level links 

y / and second level links, (creating a hierarchical datij j^ruc ture'Tbat associates the 

text and links to the categories, the first-level links and the se cond-level links; 
generating an audio representation of at least a portion of the parsed electronic 
document, the audio representation being communicated to a client; and 



PROPOSED AMENDED CLAIMS - DO NOT ENTER IN RECORD 
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prompting, the user t o select a category from the h ierarchical data structur e and 
then suc cessively prompting the user to select any first-level links a n d second-level links, 
s uch that t.lic. content of the electronic docum ent is presented audib l y to the us e r and the 
can ma ke verbal selections . 



PROPOSED AMENDED CLAIMS - DO NOT ENTER IN RECORD 
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COMBINED AMENDMENT & PETITION FOR EXTENSION OF 
TIME UNDER 37 CFR^ ^136(3) (Small Entity) 



Docket No. 
14999.3 



In Re Application Of: Weseman* 



M 2 3 2001 



Serial No. 


filing Date ^ 


Examiner 


Group Art Unit 


09/464,989 


Decenfe«f*#9 


N. Putt 


2645 



Invention: VOICE INTERFACE FOR ELECTRONIC DOCUMENTS 



RECEIVEC 
JUL 3 1 2001 
Technology center 2 



TO THE ASSISTANT COMMISSIONER FOR PATENTS: 



This is a combined amendment and petition under the provisions of 37 CFR 1.1 36(a) to extend the period for filing a 
response to the Office Action of February 23, 2001 j n the above-identified application. 

The requested extension is as follows (check time period desired): 

□ One month E9 Two months □ Three months □ Four months □ Five months 
from: May 23, 2001 until: July 23, 2001 



A verified statement of small entity status as a small entity under 37 CFR 1.27: 
□ is enclosed. 

Kl has already been filed in this application. 
The fee for the amendment and extension of time has been calculated as shown below: 



TOTAL CLAIMS 
INDEP. CLAIMS 



CLAIMS REMAINING 
AFTER AMENDMENT 



CLAIMS AS AMENDED 



HIGHEST # 
PREV. PAID FOR 



NUMBER EXTRA 



ADDITIONAL 



FEE FOR AMENDMENT 



FEE FOR EXTENSION OF TIME 



TOTAL FEE FOR AMENDMENT AND EXTENSION OF TIME 



$195.00 



COMBINED AMENDMENT & PETITION FOR EXTENSION OF 
TIME UNDER 37 CFR 1.136(a) (Small Entity) 



■««. 2 3 200J 



Docket No. 
14999.3 



HbCEIVEB- 

JUL 3 1 2001 
Technology Center 2600 



The fee^ftaafetftWnendment and extension of time is to be paid as follows: 
Kl A check in the amount of $195.00 for the amendment and 



of time is enclosed, 
the amount of S195.00 



Please charge Deposit Account No. 23-3178 
A duplicate copy of this sheet is enclosed. 

The Commissioner is hereby authorized to charge payment of the following fees a; 
communication or credit any overpayment to Deposit Account No. 
A duplicate copy of this sheet is enclosed. 
G3 Any additional filing fees required under 37 C.F.R. 1.16. 
Kl Any patent application processing fees under 37 CFR 1 . 1 7. 

If an additional extension of time is required, please consider this a petition therefor and charge any additional fees 
which may be required to Deposit Account No. 23-3178. A duplicate copy of this sheet is enclosed. 



Signature 

R. Burns Israelsen, Esq. 

Registration No. 42,685 

WORKMAN, NYDEGGER & SEELEY 

1000 Eagle Gate Tower 

60 East South Temple Street 

Salt Lake City, Utah 84111 

Telephone: 801-533-9800 

Facsimile: 801-328-1707 



Dated: July 23, 2001 



■e of Person Mailing Correspond 
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Invention: VOICE' 



fRFACE FOR ELECTRONIC DOCUMENTS 



RECEIVEC 

JUL 3 1 2001 



TO THE ASSISTANT COMMISSIONER FOR PATENTS: 



This is a combined amendment and petition under the provisions of 37 CFR 1.136(a) to exten 
response to the Office Action of February 2 3, 2001 j n the above-identified application. 

The requested extension is as follows (check time period desired): 

□ One month H Two months □ Three months □ Four months 
from: May 23, 2001 until: July 23, 2001 

A verified statement of small entity status as a small entity under 37 CFR 1 .27: 
□ is enclosed. 

(§3 has already been filed in this application. 
The fee for the amendment and extension of time has been calculated as shown below: 
CLAIMS AS AMENDED 



Technology Center 

d the period for filing a 
□ Five months 



CLAIMS REMAINING 
AFTER AMENDMENT 



HIGHEST* 
^REV. PAID FOR 



TOTAL CLAIMS 



NUMBER EXTRA 
CLAIMS PRESENT 



FEE FOR AMENDMENT 



FEE FOR EXTENSION OF TIME 



TOTAL FEE FOR AMENDMENT AND EXTENSION OF TIME 



COMBINED AMENDMENT & PETITION FOR EXTENSION OF 
JUL 1 3 2M1 



Docket No. 
14999.3 



The fee for the amendment and extension of time is to be paid as follows: 



A check in the amount of 



RECEIVED 
JUL 3 1 2001 
Technology Center 2600 



Please charge Deposit Account No. 23-3178 in the amount of $195.00 

A duplicate copy of this sheet is enclosed. 

The Commissioner is hereby authorized to charge payment of the following fees associated with this 
communication or credit any overpayment to Deposit Account No. 
A duplicate copy of this sheet is enclosed. 

Any additional filing fees required under 37 C.F.R. 1.16. 

Any patent application processing fees under 37 CFR 1.17. 



If an additional extension of time is required, pies 
which may be required to Deposit Account No. 



a petition therefor and charge any additional fees 
A duplicate copy of this sheet is enclosed. 



Dated: July 23, 2001 



urns Israelsen, Esq. 
Registration No. 42,685 
WORKMAN, NYDEGGER & SEELEY 
1000 Eagle Gate Tower 
60 East South Temple Street 
Salt Lake City, Utah 84111 
Telephone: 801-533-9800 
Facsimile: 801-328-1707 



ass mail under 37 C.F.R. 1.8 and is addressed to the 
Assistant Commissioner for Patents, Washington, D.C. 
20231. 
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PATENT APPLICATION 
Docket No: 14999.3 



.JrlN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
In re application of 



Serial No.: 
Confirmation No. : 
Filed: 
For: 



Wesemann et al. 
09/464,989 



December 16, 1999 



VOICE INTERFACE FOR 
ELECTRONIC DOCUMENTS 



AMENDMENT "A" 



Art Unit 
2645 



RECEIVED 
JUL 3 1 zooi 

Technology center 2600 



Assistant Commissioner for Patents 
Washington, D. C. 20231 

Sir: 

Responsive to the Office Action of February 23, 2001, Applicants respectfully request 
entry of the following amendments and reconsideration of the pending claims in view of the 
amendments, the matters discussed at the interview, the remarks herein, and an accompanying 
Petition for Extension of Time that extends the period of response by two months. 



IN THE SPECIFICATION 

Please replace the paragraph beginning at page 20, line 22 with the following rewritten 
paragraph: 



1 



In contrast to Figure 5, Figure 1 1 is a flow chart illustrating the operation 
of a preferred embodiment of the present invention that provides only the default 
mapping of separating text and links. In step 1110, a client selects the option of 
starting with a list of favorite Internet sites. Text to Speech 420 prompts the user 
J to select one of the sites listed, step 1 120. As before, in step 1 130 the document is 

retrieved using the protocols that are appropriate given the document's location. 
Again, as before, the content is parsed in step 1140 to identify any title, any links, 
any link names, and any text included in the document. 



IN THE CLAIMS 

Please cancel claims 3, 17-40, 42, 43, 48-50, 52 and 57 without prejudice. 
Please ame\d claims 1, 4, 6, 16, 41, 45-47, 51, 53-56 and 58-60 as follows: 



1. (Amended) In a system that includes an information service and a telephone 
for interacting with the information service, a method of enabling a user of the telephone to 
access and navigate electronic documents by presenting to the user an audio representation of a 
hierarchy of links of the document so as to enhance the ability of the user to navigate the 



#3, 

electronic documents, the method comprising the acts of: 



obtaining an electronic document; 

parsing the electronic document to identify any text and any links included in the 
content of the electronic document; 

mapping content of the parsed electronic document by performing the acts of: 



A3, 



4* 



determining whether the text and links included in the content of the 
document represent categories, first-level links and second-level links in a 
hierarchical relationship one with another; and 

to the extent that die text and links represent categories, first-level links 
and second level links, creating a. hierarchical data structure that associates the 
text and links to the categories, the first-level links and the second-level links; 
generating an audio representation of at least a portion of the parsed electronic 
document, the audio representation being communicated to a client; and 

prompting the user to select a category from the hierarchical data structure and 
then successively prompting the user to select any first-level links and second-level links, 
such that the content of the electronic document is presented audibly to the user and the 
' can make verbal selections. — — 



A3 



3 

ft (Amended) A method as recited in claim 1 further comprising the acts of 
generating an audio representation of the one or more categories. ; 



ft. (Amended) A method as recited in claim 1 wherein the act of mapping 
includes creating a hierarchy of said any text and said any links. 



]A. (Amended) A method as recited in claim 1 further comprising the act of 
0^ providing the user with a visual representation of the electronic document simultaneously with 
the act of generating the audio representation. '. 



(Amended) In an information system that has access, through the Internet, t< 



electronic documents, wherein the electronic documents are written in a markup language having 
/>- elements that onfy-afe-eapable of describing the visual appearance of the electronic documents, 
and the information system is connected to a client by a telephony infrastructure, a method for 
providing the user with telephone access to the electronic documents, the method comprising the 
acts of: 

obtaining an electronic document through the Internet; 

parsing the electronic document to identify any text and any links included in the 
^/^ content of the electronic document; 

mapping content of the parsed electronic document by performing the acts of: 

determining whether the text and links included in the content of the 
document represent categories, "first-level links and second-level links in a 
hierarchical relationship one with another; and 

to the extent that the text and links represent categories, first-level links 
and second level links, creating a hierarchical data structure that associates the 
text and links to the categories, the first-level links and the second-level links; 
prompting the client to traverse the hierarchical data structure, including 
automatically selecting any category and any associated link when exactly one category 
and exactly one associated link is present; 

receiving input from the client to traverse the hierarchical data structure, wherein 
the input includes instructions for selecting an electronic document with text; and 



generating an audio representation of the parsed electronic document, including a 
count of any links and any text present in the electronic document, the audio 
representation being communicated to the client. 



^4. (Amended) A method as recited m claim ^i, wherein the instruction is spoken 
o the telephone. 



A 

yi, wher 



(Amended) A method as recited in claim >4, wherein the input further includes 



at least one of an instruction to email, fax, or voice mail at least a portion of the electronic 
document to a third party selected from a contact list of the client. 

(Amended) A method as recited in claim jA, wherein the audio representation 
includes information representing at least one link and wherein the input includes an instruction 
to select the at least one link, the method further comprising the act of following the at least one 



$fl. (Amended) A method as recited m claim ^4 further comprising the act of 
providing the user with a visual representation of the electronic document simultaneously with 
the act of generating the audio representation. 



(Amended) A system providing access, through an audio interface, to 
electronic documents so as to enable a user of the audio interface to access and navigate the 
^electronic documents by presenting to the user a hierarchy of links of the ddcument-the-system 
comprising: 

processor means for obtaining an electronic document; 

processor means for parsing the electronic document to identify any text and any 
links included in the content of the electronic document; 

processor means for mapping content of the parsed electronic document by 
performing the acts of: 

determining whether the text and links included in the content of the 

document represent categories, first-level links and second-level links in a 

hierarchical relationship one with another; and 

to the extent that the text and links represent categories, first-level links 

and second level links, creating a hierarchical data structure that associates the 

text and links to the categories, the first-level links and the second-level links; 

processor means for generating an audio representation of, at least a portion of the 
parsed electronic document, the audio representation being communicated to a client; and 

processor means for prompting the user to select a category from the hierarchical 
data structure and then successively prompting the user to select any first-level links and 
second-level links, such that the content of the electronic document is presented audibly 
/Q, to the user and thj^can make verbal selections. 



— 7f 

i claim *4\ furtl 



I (Amended) A system as recited in claim <6 fu rther comprising processor 

means for receiving spoken input from the user. 

^4. (Amended) A system as recited in claim ^5 wherein the input comprises at 
least one of an instruction to email, fax, or voice mail at least a portion of the electronic 
document to a third party selected from a contact list of the client. 

ok i r 

(Amended) A system as recited in claim $^ wherein the audio representation 
includes information representing at least one link and wherein the input includes an instruction 
to select the at least one link, the system further comprising processor means for following the at 
least one link. 



,/>6. (Amended) A computer program product for providing access, through an 
audio interface, to electronic documents so as to enable a user of the audio interface to access 
and navigate the electronic documents by presenting to the user a hierarchy of links of the 
fly de©w*se»t-the-computer program product comprising: 

a computer-readable medium carrying computer-executable instructions for 
implementing the method, the computer-executable instructions comprising: 
program code means for obtaining an electronic document; 
program code means for parsing the electronic document to identify any 
text and any links included in the content of the electronic document; 

program code means for mapping content of the parsed electronic 
document by performing the acts of: 

determining whether the text and links included in the content of 
the document represent categories, first-level links and second-level links 
in a hierarchical relationship one with another; and 

to the extent that the text and links represent categories, first-level 
links and second level links, creating a hierarchical data structure that 
associates the text and links to the categories, the first-level links and the 
second-level links; 

program code means for generating an audio representation of at least a 
portion of the parsed electronic document, the audio representation being 
communicated to a client; and 

program code means for prompting the user to select a category from the 
hierarchical data structure and then successively prompting the user to select any 
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55s. (Amended) A computer-readable medium as recited in claim $« wherein the 
computer-executable instructions further comprises program code means for receiving spoken 
input from the user. 

59. (Amended) A computer-readable medium as recited in claim ^8 wherein the 
input comprises at least one of an instruction to email, fax, or voice mail at least a portion of the 
electronic document to a third party selected from a contact list of the client. 

/ / 

60. (Amended) A computer-readable medium as recited in claim 58 wherein the 
audio representation includes information representing at least one link and wherein the input 
includes an instruction to select the at least one link, wherein the computer-executable 
instructions further comprise program code means for following the at least one link. 



REMARKS 

Applicants express appreciation to the Examiner for the Interview of April 17, 2001 
conducted with Applicants' attorneys. The Office Action of February 23, 2001 rejected claims 
1-60. By this paper, claims 3, 17-40, 42, 43, 48-50, 52 and 57 have been canceled. Accordingly, 
claims 1, 2, 4-16, 41, 44-47, 51, 53-56 and 58-60 are pending. Applicants respectfully request 
favorable reconsideration of the pending claims in view of the amendments made herein and the 
matters discussed at the interview. 

The Office Action rejected claims 1-7, 9-23, 25-33, 35-44, 46-53, 55-58 and 60 under 35 
U.S.C. § 102(e) as being anticipated by U.S. Patent No. 5,884,262 to Wise. Claims 8, 11, 24, 34, 
36, 45, 54 and 59 were rejected under 35 U.S.C. § 103(a) as being unpatentable over Wise in 
view of U.S. Patent No. 5,953,392 to Rhie. 

At the Interview of April 17, 2001, Applicants' attorney discussed proposed amendments 
to claim 1. Claim 1 as amended now recites: 

mapping content of the parsed electronic document by performing the acts 

of: 

determining whether the text and links included in the .content of 
the document represent categories, first-level links and second-level links 
in a hierarchical relationship one with another; and 

to the extent that the text and links represent categories, first-level 
links and second level links, . creating a hierarchical data structure that 
associates the text and links to the categories, the first-level links and the 
second-level links; 

As discussed at the interview, neither Wise nor Rhie teach or suggest mapping content of 
a parsed electronic document in the foregoing manner. In particular, Wise and Rhie fail to teach 
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or suggest "determining whether the text and links included in the content of the document 
represent categories, first-level links and second-level links in a hierarchical relationship one 
with another" and farther fail to teach or suggest "creating a hierarchical data structure that 
associates the text and links to the categories, the first-level links and the second-level links." 

Mapping the content of a parsed electronic document as recited in claim 1 as amended 
can provide certain benefits not understood in the prior art, including enhancing the ability of the 
user to navigate electronic documents. Moreover, claim 1 as amended recites: 

prompting the user to select a category from the hierarchical data structure 
and then successively prompting the user to select any first-level links and 
second-level links, such that the content of the electronic document is presented 
audibly to the user and the can make verbal selections 

As further discussed at the Interview, none of the cited references prompt a user to select 
from a hierarchical structure in the foregoing manner. Applicants respectfully submit that claim 
1 as amended distinguishes from the cited references for the foregoing reasons. 

The other independent claims 41, 51 and 56 include a limitation substantially similar to 
the act of "mapping content of the parsed electronic document . . ." discussed above in reference 
to claim 1. In addition, claims 51 and 56 include a limitation substantially similar to the act of 
"prompting the user to select a category from the hierarchical data structure . . ." discussed above 
in reference to claim 1. Applicants respectfully submit that claims 41, 51 and 56 also distinguish 
from the cited references for at least these reasons. 
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The other pending claims are dependent claims that depend from the independent claims 
discussed above and are patentable at least on the basis of their dependency from a patentable 
base claim. 

Applicants also point out that several of the claims have been amended to promote 
clarity, to provide terminology that is consistent with that of the base claims, and for other 
reasons that are not related to either responding to a rejection of the claims or distinguishing 
from cited art. 

Attached hereto is a marked-up version of the changes made to the previous version of 
the specification and claims by this amendment. The attached pages are captioned "VERSION 
WITH MARKINGS TO SHOW CHANGES MADE." 

For the foregoing reasons, Applicants submit that the pending claims are in condition for 
allowance and courteously request favorable action. If there are any outstanding issues that 
could be resolved by telephone, the Examiner is invited to contact the undersigned attorney. 

Dated this 23 rd day of July, 2001. 

Respectfully submitted, 

R. BURNS ISRAELSEN 
Attorney for Applicant 
Registration No. 42,685 

WORKMAN, NYDEGGER & SEELEY 
1000 Eagle Gate Tower 
60 East South Temple 
Salt Lake City, Utah 841 11 
Telephone: (801) 533-9800 
Facsimile: (801) 328-1707 

RBLllr 
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VERSION WITH MARKINGS TO SHOW CHANGES MADE 



IN THE SPECIFICATION 

At page 21, line 2, "in" has been replaced with -is--. 
IN THE CLAIMS 

Claims 3, 17-40, 42, 43, 48-50, 52 and 57 have been canceled. 

Claims 1, 4, 6, 16, 41, 45-47, 51, 53-56 and 58-60 have been amended as follows: 

1. (Amended) In a system that includes an information service and a telephone 
[an input/output device] for interacting with the information service, a method of enabling a user 
of the telephone to access and navigate [providing an audio interface for accessing] electronic 
documents by presenting to the user an audio representation of a hierarchy of links of the 
document so as to enhance the ability of the user to navigate the electronic documents , the 
method comprising the acts of: 

obtaining an electronic document; 

parsing the electronic document to identify any text and any links included in the 
content of the electronic document; [and] 

mapping content of the parsed electronic document by performing the acts of: 

determining whether the text and links included in the content of the 

document represent categories, first-level links and second-level links in a 

hierarchical relationship one with another; and 

to the extent that the text and links represent categories, first-level links 

and second level links, creating a hierarchical data structure that associates the 

text and links to the categories, the first-level links and the second-level links; 

generating an audio representation of at least a portion of the parsed electronic 
document, the audio representation being communicated to a client ; and 

prompting the user to select a category from the hierarchical data structure and 
then successively prompting the user to select any first-level links and second-level links, 
such that the content of the electronic document is presented audibly to the user and the 
can make verbal selections . 
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4. (Amended) A method as recited in claim 1 [3] further comprising the acts of 
generating an audio representation of the one or more categories. 

6. (Amended) A method as recited in claim 1 [3] wherein the act of mapping 
includes creating a hierarchy of said any text and said any links. 

16. (Amended) A method as recited in claim 3 further comprising the act of 
providing the user with a visual representation of the electronic document simultaneously with 
the act of generating [communicating] the audio representation. 
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41. (Amended) In an information system that has access, through the Internet, to 
[one or more] electronic documents, wherein the [one or more] electronic documents are written 
in a markup language having elements that only are capable of describing the visual appearance 
of the [one or more] electronic documents, and the information system is connected to a client by 
a telephony infrastructure, a method for providing the user with telephone access to the [one 
more] electronic documents, the method comprising the acts of: 

obtaining an [one of the one or more] electronic document through the Internet 

[documents]; 

parsing the [one] electronic document to identify any text and any links included 
in the content of the [one] electronic document; [and] 

mapping content of the parsed electronic document by performing the acts of: 

determining whether the text and links included in the content of the 

document represent categories, first-level links and second-level links in a 

hierarchical relationship one with another; and 

to the extent that the text and links represent categories, first-level links 

and second level links, creating a hierarchical data structure that associates the 

text and links to the categ ories, the first-level links and the second-level links; 

prompting the client to traverse the hierarchical data str ucture, including 
automatically selecting any category and any associated link when exactly one category 
and exactly one associated link is present; 

receiving input from the client to traverse the hierarchical data structure, wherein 
the input includes instructions for selecting an electronic document with text; and 

generating an audio representation of [at least a portion of] the parsed [one] 
electronic document, including a count of any links and any text present in the electronic 
document, the audio representation being communicated to the client. 

44. (Amended) A method as recited in claim 41 [further comprising the act of 
receiving an instruction from the client], wherein the instruction is spoken into the telephone. 

45. (Amended) A method as recited in claim 44 2 wherein the input further includes 
[instruction received from the client is] at least one of an instruction to email, fax, or voice mail 
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at least a portion of the electronic document to a third party selected from a [the client's] contact 
list of the client . 

46. (Amended) A method as recited in claim 44 2 wherein the audio representation 
includes information representing at least one link and wherein the input includes an instruction 
[from the client is] to select the at least one link, the method further comprising the act of 
following the at least one link. 

47. (Amended) A method as recited in claim 41 further comprising the act of 
providing the user with a visual representation of the electronic document simultaneously with 
the act of generating [communicating] the audio representation. 
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51. (Amended) A system providing access, through an audio interface, to 
electronic documents[,] so as to enable a user of the audio interfac e to access and navigate the 
electronic documents bv presenting to the user a hie rarchy of links of the document [wherein the 
electronic documents are written in a markup language having elements that only are capable of 
describing the visual appearance of the electronic documents,] the system comprising: 
processor means for obtaining an electronic document; 

processor means for parsing the electronic document to identify any text and any 
links included in the content of the electronic document; [and] 

processor means for mapping content of the parsed electronic doc ument by 
performing the acts of: 

determining whether the text and links included in the , content of the 
doc ument represent categories, first-level links and seco nd-level links in a 
hierarchical relationship one with another; and 

to the extent that the text and links represent categories, f irst-level links 
and second level links, creating a hierarchical data structure that associates the 
text and links to the categories, the first-level links and the second-level links; 
processor means for generating an audio representation of at least a portion of the 
parsed electronic document, the audio representation being communicated to a client^and 
processor means for prompting the user to select a category from the hiera rchical 
data structure and then successively prompting the user to select any first-level l inks and 
second-level links, such that the content of the electronic d ocument is presented audibly 
to the user and the can make verbal selections . 

53. (Amended) A system as recited in claim 51 further comprising processor 
means for receiving [a] spoken input [instruction] from the user [client]. 

54. (Amended) A system as recited in claim 53 wherein the input comprises 
[instruction received from the client is] at least one of an instruction to email, fax, or voice mail 
at least a portion of the electronic document to a third party selected from a [the client's] contact 
list of the client . 
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55. (Amended) A system as recited in claim 53 wherein the audio representation 
includes information representing at least one link and wherein the input includes an instruction 
[from the client is] to select the at least one link, the system further comprising processor means 
for following the at least one link. 
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56. (Amended) A computer program product for providing access, through an 
audio interface, to electronic documents[,] so as to enable a user of the audio interface to access 
and navigate the electronic documents by presenting to the user a hierarchy of links of the 
document [wherein the electronic documents are written in a markup language having elements 
that only are capable of describing the visual appearance of the electronic documents,] the 
computer program product comprising: 

a computer-readable medium carrying computer-executable instructions for 
implementing the method, the computer-executable instructions comprising: 
program code means for obtaining an electronic document; 
program code means for parsing the electronic document to identify any 
text and any links included in the content of the electronic document; [and] 

program code means for mapping content of the parsed electronic 
document by performing the acts of: 

determining whether the text and links included in the content of 
the document represent categories, first-level links and second-level links 
in a hierarchical relationship one with another; and 

to the extent that the text and links represent categories, first-level 
links and second level links, creating a hierarchical data structure that 
associates the text and links to the categories, the first-level links and the 
second-level links; 

program code means for generating an audio representation of at least a 
portion of the parsed electronic document, the audio representation being 
communicated to a clien t; and 

program code means for prompting the user to select a category from the 
hierarchical data structure and then successively prompting the User to select any 
first-level links and second-level links, such that the content of the electronic 
document is presented audibly to the user and the can make verbal selections . 

58. (Amended) A computer-readable medium as recited in claim 56 wherein the 
computer-executable instructions further comprises program code means for receiving [a] spoken 
input [instruction] from the user [client]. 
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59. (Amended) A computer-readable medium as recited in claim 58 wherein the 
input compri ses [instruction received from the client is] at least one of an instruction to email, 
fax, or voice mail at least a portion of the electronic document to a third party selected from a 
[the client's] contact list of the client . 

60. (Amended) A computer-readable medium as recited in claim 58 wherein the 
audio representation includes information representing at least one link and wherein the input 
includes an instruction [from the client is] to select the at least one link, wherein the computer- 
executable instructions further comprise [comprises] program code means for following the at 
least one link. 
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Art Unit: 2645 

DETAILED ACTION 

1 . This action is in response to applicant's amendment filed on July 23, 2001 . Claims 1,2,4- 
16,41,44-47,51,53-56,58-60 are now pending in the present application. 

EXAMINER'S AMENDMENT 

2. An examiner's amendment to the record appears below. Should the changes and/or 
additions be unacceptable to applicant, an amendment may be filed as provided by 37 CFR 

1.3 12. To ensure consideration of such an amendment, it MUST be submitted no later than the 
payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview with R. 
Burns Israelsen on September 21, 2001. 

The application has been amended as follows: 

In claim 1, line 21, before "can" inSfert — user- 

In claim 1 1, line 2, delete Jany" 

In claim 12, line 2, delete "any" (both occurrences) 

In claim 14, line 2 change "only are" to -are only- 



In claim 41, line 3, channel "only are" to -are only- 

In claim 51, lines 3, chaf/ge "documenythe" to -document, the- 

In claim 51, line 21, before "can" insert -user- 

In claim 56, line 4, chanp: "document the" to -document, the- 

In claim 56, line 25, before "can" ins/rt -user- 
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Drawings 

3 . The Drawings submitted on December 1 6, 1 999 have been approved by the Draftsman. 

REASONS FOR ALLOWANCE 

4. The following is an examiner's statement of reasons for allowance: 

The prior art of record fails to disclose or render obvious, alone or in combination, claims 
1,2,4-16,41,44-47,51,53-56,58-60, (renumbered 1-28). The subject claims are directed to a 
system, method and computer program product for enabling a user to access and navigate 
electronic documents via a telephone. 

Specifically regarding claims 1,41,51 and 56, the prior art of record does not teach or 
suggest the feature of determining whether the text and links which was parsed represent 
categories, first-level links and second-level links in a hierarchical relationship with each other 
and creating a hierarchical data structure that associates the text and links to the categories, the 
first-level links and the second-level links; and prompting the user to select a category from the 
hierarchical data structure. 

The previously applied prior art of record Wise et al. US Patent 5,884,262 teaches a 
system and method for accessing electronic documents via a telephone. The system of Wise 
parses the electronic documents to determine the content of the document which includes 
determining the headers, labels, text graphics, audio information, comments and other types of 
content. Wise, however, fails to teach of creating a hierarchical relationship of the content of the 
electronic document with each other. 

The prior art of record Rhie et al. US Patent 5,953,392 teaches a system for accessing 
and browsing the Internet through the use of a telephone. The system performs text to speech 
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translations of the text on the page. Rhie, however fails to teach or suggest of determining 
whether the text and links represent categories, first-level links and second-level links in a 
hierarchical relationship with each other and creating a hierarchical data structure that associated 
the text and links to the categories, the first-level links and the second-level links; and prompting 
the user to select a category from the hierarchical data structure. 

The remaining prior art of record fails to teach the above feature or provide a motivation 
to add such a feature. 

Any comments considered necessary by applicant must be submitted no later than the 
payment of the issue fee and, to avoid processing delays, should preferably accompany the issue 
fee. Such submissions should be clearly labeled "Comments on Statement of Reasons for 
Allowance." 

Conclusion 

5. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Ball et al. US Patent 6,240,391 Bl teaches of a method for assembling and presenting a 
structured electronic message. Ball however fails to teach of creating a hierarchical relationship 
of the content of the electronic message. 

Saylor et al. US Patent 6,263,051 Bl teaches of a method of providing a user an 
electronic message which will allow a user to navigate though a series of categories in 
hierarchical relationship with each other. Saylor however fails to teach of parsing a document to 
determine the content of the electronic page and creating a hierarchical relationship based on the 
parsing. 
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Ladd et al. US Patent 6,269,336 Bl teaches of a markup language to provide interactive 
service. Ladd however, fails to teach or suggest of determining whether the text and links 
represent categories, first-level links and second-level links in a hierarchical relationship with 
each other and creating a hierarchical data structure that associated the text and links to the 
categories, the first-level links and the second-level links; and prompting the user to select a 
category from the hierarchical data structure. 

6. Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D.C. 20231 

or faxed to: 

(703) 872-9314, (for formal communications intended for entry) 

Or: 

(703) 872-9314, (for informal or draft communications, please label 
"PROPOSED" or "DRAFT") 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal 
Drive, Arlington. VA, Sixth Floor (Receptionist). 

7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ovidio Escalante whose telephone number is (703) 308-6262. 
The examiner can normally be reached on Monday to Friday from 6:30 AM to 5:00 PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Fan Tsang, can be reached on (703) 305-4895. The fax phone number for this Group 
is (703) 872-9314. 

Communications via Internet e-mail regarding this application, other than those under 35 
U.S.C. 132 or which otherwise require a signature, may be used by the applicant and should be 
addressed to [fan.tsang@uspto.gov]. 

All Internet e-mail communications will be made of record in the application file. PTO 
employees do not engage in Internet communications where there exists a possibility that 
sensitive information could be identified or exchanged unless the record includes a properly 
signed express waiver of the confidentiality requirements of 35 U.S.C. 122. This is more clearly 
set forth in the Interim Internet Usage Policy published in the Official Gazette of the Patent and 
Trademark on February 25, 1997 at 1195 OG 89. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the Technology Center 2600 Customer Service Office whose telephone 
number is (703) 306-0377. 

Ovidio Escalante 
Examiner 
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Applicants respectfully submit that the claimed invention as set forth in each of the 

wmmsm 

over the prior art of record. 

DatedtHsl<_dayofJk££^ .2001- 

Respectfully submitted, 

R. BURNS ISRAELSEN 
Attorney for Applicant 
Registration No. 42,685 
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Guide to Filing a Utility Patent Application-' 



INTRODUCTION 




The U, S. Patent and Trademark Office (PTO) is the government agency responsible for examining 
patent applications and issuing patents. A patent is a type of property right It gives the patent holder 
the right, for a limited time, to exclude others from making, using, or selling the subject matter that is 
within the scope of protection granted by the patent. The PTO determines wether a patent should be 
granted in particular case. However, it is up to the patent holder to enforce his or her own rights if 
the PTO does grant a patent. 

The purpose of this guide is to provide you with basic information about filing a utility patent 
application. A patent application is a complex legal document, best prepared by one trained to 
prepare such documents. Thus, after reviewing this guide, you may wish to consult with a patent 
attorney or agent Additional information is available: 

• by calling the PTO's General Information Services at 800-PTO-9199 or 703-308-4357, 

• from the PTO's Web site at www.uspto.gov^ and ^— 1220 

• at your nearest Patent and Trademark Depository Library (PTDL). You will find information 

• regarding the nearest PTDL at the end of this guide. ^-1230 

There are various types of patents - utility, design, and plant. There are also two types of utility 
patent application ~ provisional and nonprovisional. Each year the PTO receives approximately 
200,000 patent applications. Most of these are for nonprovisional utility patents. 

This guide contains information to assist you in filing your nonprovisional utility patent application. 
It discusses the required parts of the utility patent application and includes samples of some of the 
forms you may use. This information is generally derived from the Patent Act, found at Title 35 of 
the United States Code (U.S.C.), and Title 37 of the Code of Federal Regulations (CFR), These 
materials are available atPTDLs and at most law libraries. 
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